Energy dispatchcontrollersforaphotovoltaicsystem


a b s t r a c t
In thispapertwoenergydispatchcontrollersforuseinagrid-independentphotovoltaic(PV)systemare
presented.Thefirst,anoptimalenergydispatchcontroller,isbasedonaclassofAdaptiveCriticDesigns
(ACDs)calledActionDependentHeuristicDynamicProgramming(ADHDP).ThisclassofACDsusestwo
neuralnetworkstoevolveanoptimalcontrolstrategyovertime.Thefirstneuralnetworkor‘‘Action’’
networkdispensestheactualcontrolsignalswhilethesecondnetworkor‘‘Critic’’networkusesthese
controlsignalsalongwiththesystemstatestoprovidefeedbacktotheactionnetwork,measuring
performanceusingautilityfunction.Thisfeedbackloopallowstheactionnetworktoimprovebehavior
over time.Theoptimalenergydispatcherplacesemphasisonalwaysmeetingthecriticalload,followed
by keepingthechargeofthebatteryashighaspossiblesoastobeabletopowerthecriticalloadin
cases ofextendedlowoutputfromthePVarray,andlastlytopowerthenon-criticalloadinsofarasto
not interferewiththefirsttwoobjectives.Thesecondenergydispatchcontrollerisasmartenergy
dispatchcontrollerandisbuiltusingknowledgefromanexpert,codifiedintoaseriesofstaticrules.
This smartenergydispatchcontrolleriscalledthe‘‘PV-priority2’’controller.Theseenergydispatchers
are comparedwithastaticschemecalledthe‘‘PV-priority1’’.ThePV-priority1controllerrepresents
the standardcontrolstrategy.ResultsshowthattheADHDP-basedoptimalenergydispatcher(or
controller)outperformsthestandardPV-priority1energydispatcherinmeetingthestatedobjectives,
but trailsthePV-priority2energydispatcher.However,themajoradvantageoftheADHDPcontrolleris
that noexpertisrequiredfordesigningthecontroller,whereasforarule-basedcontrollersuchasthe
PV-priority2controller,anexpertisalwaysrequired.







1. Introduction
As thecostsoffossilfuelscontinuetorise,itisbecoming
economically importanttoinvestigateothersourcesofenergy.
Additionally, increasedcustomerdemandsandloadsarebegin-
ning tooutstripthegrid’scapacitytoservetheseloads.Assuch,
distributed generationofalternativeenergysourcesisbecominga
very heatedresearchanddevelopmentarea.
Currently, thereareseveralalternativeenergysourcesavail-
able: wind,solar,hydro-electricandgeo-thermal,tonameafew.
Hydro-electric andgeo-thermalplantsoftenrequirelargefoot
prints thatareatoddswithdevelopedareasandenvironmental
considerations. Windenergyiscurrentlyenjoyingveryenergetic
growth, ataround24%peryearsincetheyear2000intheUS(US
Department ofEnergy’sOfficeofEnergyEfficiencyAndRenewable
Energy, 2007). But,windpowerhasitsdownsidesaswellsince
not alllocationsreceiveenoughsustainedwindstobeproductive,
and eventhenproductionissometimessporadic.Ofallofthe
mentioned sources,solarpowerseemstobethemostpromising
in thatalllocationsonEarthreceivepredictablesunlighttosome
degree, andthesolararraysusedtoconvertsunlightinto
electricity (photovoltaicorPVarrays)scaleverywellfromvery
small sizesforcalculatorstoverylargesizesusedincentralized
power plants.
AnotherbenefitofsolarenergyisthatthePVarrayscontainno
movingpartsandcanlastseveraldecadesbeforeneedingtobe
replaced.Duringthistime,theonlymaintenancethatmayneedtobe
done toPVarraysisdustorsnowclearingandcheckingforalignment
problems.AndwhilethePVarraysarewithintheirratedlifetimes,
they generallyperformreliablywhiletheSunisshining.
Even withtheseadvantages,thereisonemajordrawbackofPV
systems thatlimitstheiradoptionrateandthatisthecostofthese
systems. Thecostofenergyderivedfromthesesystemsmakes
them notcurrentlycompetitivewithothersources.However,due
to technological,manufacturingandresourceimprovements,
these costshavesteadilyfalleninpreviousyears(Messenger
and Ventre,2004) andareexpectedtocontinuetodoso.Asthe
price falls,thissourcewillbemorecompetitive.Evenso,the
payback time(theamountoftimerequiredfortheinitial
investment inaPVsystemtoequalthecostsaccruedfrom
purchasing powerthroughthegrid)canbelengthy,andinsome
cases aslongas30yearsormore.
ARTICLEINPRESS
Contents listsavailableat ScienceDirect
journalhomepage: elsevierlocate/engappai
EngineeringApplicationsofArtificialIntelligence
0952-1976/$ -seefrontmatter & 2009 ElsevierLtd.Allrightsreserved.
doi:10.1016/j.engappai.2009.11.001
n Corresponding author.Tel.:+15733416641;fax:+15733414532.
E-mail address: gkumar[at]ieee.org (G.K.Venayagamoorthy).
Engineering ApplicationsofArtificialIntelligence23(2010)249–261
ARTICLEINPRESS
In ordertomakePVsystemscheaper(andshortenthepayback
period), optimalcontrolcanbeusedtomoreeffectivelyutilize
energy generatedbythePVarray,resultinginsmallerrequired
batteries andarrayswhilestillmaintainingthecriticalload.This
reduction insystemcomponentsizeleadstoadirectreductionin
cost fortheentiresystem.
The traditionalenergydispatcherforaPVsystemiscalledthe
‘‘PV-priority’’ (called‘‘PV-priority1’’inthispaper)controlscheme
(Henze andDodier,2003) andwillfirstattempttopowerallloads
using energyfromthePVarray;ifthereisnotenoughenergy
available fromthePVarraythenenergyfromthebatteryisused
to makeuptheshortfall(ifavailable),andifthereismoreenergy
available fromthePVarraythenthebatteriesarechargedwith
the difference(ifpossible).
In thispaper,twoadditionalcontrollersaredeveloped:the
‘‘PV-priority 2’’andanoptimalcontrollerbasedonaclassof
adaptive criticdesigns(ACDs)calledaction-dependentheuristic
dynamic programming,orADHDP(Werbos, 1992; Venayagamoorthy
etal.,2002; Prokhorov andWunsch,1997). The‘‘PV-
priority 2’’controlschemeissimilartothePV-priority1scheme
in thatitsperformanceisrulebased,butisdifferentinthatit
attempts toalwayspowerthecriticalloadfirst,thenchargethe
battery to70%,andfinallyusewhateverenergyisavailablefrom
the PVarrayandanythingover70%stateofchargeinthebattery
to powerthenon-criticalload.
The ADHDP-basedoptimalenergydispatcherontheother
hand isnotrulebased,anddevelopsitscontrolactionstrategyby
adapting itsperformanceinresponsetoameasuredmetricvalue.
Adaptive criticdesignsuseacombinationofdynamicprogram-
ming andreinforcementlearning,andtheADHDPmethodisthe
simplest oftheACDfamily(itusesonly2neuralnetworks).One
of theneuralnetworks(calledthe‘‘action’’networkor‘‘actor’’)is
responsible forprovidingthecontrolsignalswhilethesecond
(called the‘‘critic’’network)critiquesthesecontrolsignalsover
time. Theobjectivesofthisenergydispatcherarethefollowing:
i). Completelypowerthecriticalloadatalltimes.
ii). Maintainthebatterystateofchargeashighaspossiblesoas
to beabletomeetthecriticalloadduringtimesofreduced(or
non-existent) energyfromthePVarray.
iii). Powerthenon-criticalloadsuchthatthecontrollerisstill
able tomeetthefirsttwoobjectives.
Another advantageoftheADHDP-basedcontrolleristhatsince
it isnotrulebased,itsbehaviorcanbemodifiedovertimetocope
with changingmeteorologicalconditions.Onesuchimportant
condition isthebehavioroftheSunanditsassociatedoutput.
Even thoughtheoutputingeneralremainsrelativelyconstant,
there aresmalldeviationsandatleastsomeworkhasbeen
performed intopredictingsunspotactivity(Xie etal.,2006; Day
and Nandi,2008).
Other typesofcontrollershavealsobeenreportedaswell,such
as aQ-Learning-basedcontroller(Henze andDodier,2003) and
others usingfuzzylogic(Chuku etal.,2005).
Section 2presentsthegrid-independentPVsolarenergy
system studiedinthispaper.Section3describesthePV-priority
controllers. Section4describestheADHDPoptimalcontroller
design. Section5presentstheevaluationandcomparisonofthe
ADHDP optimalPVcontrollerandthePV-prioritycontrollers
performances onTypicalMeteorologicalYear2(TMY2)data
(National RenewableEnergyLaboratory,1995) ofsixcitiesinthe
United StatesofAmerica.Finally,theconclusionisgivenin
Section 6.
2. Grid-independentPVsolarenergysystem
The completephotovoltaicsystemmodeliscomposedofthe
PV array,maximumpowerpointtracker,controller,battery
charge controller,batteries,inverter,criticalloadsandnon-critical
loads. Thecriticalloadconsistsofloadsthatshouldnotbe
dropped (suchasrefrigeration,emergencyradiocommunication),
while thenon-criticalloadcontainsitemswhicharenon-essential
(television, etc.).
In ordertosimplifythesimulationandfocusonthecontroller
aspect ofthissystem,allofthesupportingsystemcomponents
(such astheinverter,maximumpowerpointtracker,wiring,etc.),
are assumedtooperateat100%efficiency.Intherealworld,
however, thesedevicesdonotoperateat100%efficiencybutnew
algorithms forthemaximumpowerpointtracker(Kim etal.,
2006; Kwon etal.,2006; Mutoh, etal.,2006; Park etal.,2006;
Xiao etal.,2006) aimtoincreasetheeffectiveefficiencyofthis
Critical Load
Constant 0.124 kW
Non-Critical Load
Variable 0 kW to 0.365 kW
Battery Storage
34.56 kWh
PV Array
Energy Dispatch Controller
ξ2ECB(t)
ENCL(t)
ECL(t)
LCL(t)
LNCL(t)
ξ1EP(t)
EB(t)
Fig. 1. Diagram ofthePVsystemmodel.
Nomenclature
EB(t) energystoredinthebatteryattime t
EB,max maximum energystoredinthebattery
EPV(t) energycollectedfromthePVarrayattime t
EPV,max maximum energyabletobecollectedfromthePV
array attestlocation
EPV,wasted energy collectedfromthePVarraythatisnotbeen
used attime t
LCL(t) currentcriticalloadattime t
LCL,max maximum criticalload
LNCL(t) currentnon-criticalloadattime t
LNCL,max maximum non-criticalload
ECB(t) energydispatchedtocharge(ordischarge)thebattery
at time t
ECL(t) energydispatchedtothecriticalloadattime t
ENCL(t) energydispatchedtothenon-criticalloadattime t
Mnon-zero multiplier (usedtoinsuredivisorisnon-zero;forthis
experiment, avalueof0.1isused)
x1 efficiency ofthephotovoltaicarraysystem
x2 efficiency ofthebattery
G. KumarVenayagamoorthy,R.L.Welch/EngineeringApplicationsofArtificialIntelligence23(2010)249–261 250
ARTICLEINPRESS
device. Likewise,newenergystoragesystems(Jiang andDougal,
2006; Lemofouet andRufer,2006) mayeclipsetheperformanceof
the standardlead-acid-typebatterytechnologyprimarilyused
today. Becausethefocusofthisstudyisprimarilytoevaluatethe
performance ofthepresentedcontrolstrategies,theassumption
of 100%efficiencyofthephotovoltaicsystemismade.Ifother
efficiencies aredesired,theycanbesetbymodifyingthe
appropriate valueswithinthemodels,asdepictedin Fig. 1.
Also, thePVarrayissimulatedtobetiltedsouthatanangle
equal tothelatitudeofeachtestcityandtheefficiencyofthePV
array modelistakenas11%toaccountforvariousnon-optimal
conditions (suchasarraymisalignment,dustonthearrays,etc.).
This valueisrepresentativeofthecurrentcommerciallyavailable
range ofefficienciesforPVarrays.Generally,PVpanelsvaryin
efficiency from6%toupto30%;althoughthehighefficiency
panels aregenerallyreservedforspacecraftusagebecauseoftheir
high radiationtolerancesandhigherpower-to-weightratio.A
rough equivalenttothePVarraysbeingsimulatedinthispaper
would beanarrayofeightKyoceraKC200GTpanels.Thesepanels
are over16%efficientandwilloutput200Wduringoptimal
conditions (Kyocera, 2007). Theminimumchargeforthebattery
of 30%isrequiredtosupplyenergytotheloads(thisisconsistent
with standarddeepcyclelead-acidbatteries).
Due toinsufficientPVenergyduringwintermonthsandnoPV
energy atnight,acontrolsystemisrequiredtodecidetheamount
of energytobedispatchedtothedifferentloads,includingthe
charging ofthebattery.Thecompletesysteminschematic
diagram formisshownbelowin Fig. 1 (energy flowdepictedby
arrows).
3. PV-prioritycontrollers
3.1. PV-priority1controller
The standardcontrollercalledthe‘‘PV-priority1’’controlleris
a verysimplecontrollerwhichalwaystriestomeettheloads(the
critical andthenthenon-critical)beforechargingthebattery.At
any onetime,ifthereisnotenoughenergyfromthePVarrayto
supply theloadsthenthebalanceisdrawnfromthebattery.If
instead thereisanexcess,thenwhateverisleftoverafter
supplying theloadsisdispatchedtothebattery.Inthisway,the
controller willattempttopowerallloadsandchargethebattery
as bestitcan,withoutanyconsiderationsgiventothetime-
varying statesofthesystem.Theflowchartforthedecisionsthat
this controllermakesinanygivenhourisgivenin Fig. 2.
Use EPV to power LCL
EPV(t)=EPV(t) -L CL(t) ; LCL(t)=0
Is EPV(t) > LCL(t)?
Yes No
Use EPV(t) to power LNCL(t)
EPV(t)=EPV(t) -L NCL(t) ; LNCL(t)=0
Is EPV(t) > LNCL(t)?
Yes
No
Is EPV(t) > (EB,max(t) - EB(t))?
Yes
No
START
Use EPV(t) to charge battery
EPV(t)=EPV(t)-(EB,max(t)-EB(t)) ;
ECB(t)=EB,max(t)-EB(t);
EB(t)=EB,max(t)
Use EB(t) to power
LCL(t)
EB(t)=EB(t) -L CL(t);
ECB(t)=ECB(t) -L CL(t) ;
LCL(t)=0
Use EPV to power as much of LCL as
possible
LCL(t)=LCL(t) -E PV(t) ; EPV(t)=0
Is EB(t) > LCL(t)?
Use EPV(t) to power as
much of LNCL(t) as
possible
LNCL(t)=LNCL(t) -E PV(t) ;
EPV(t)=0
Use EB(t) to
power as much
of LCL(t) as
possible
LCL(t)=LCL(t) –
EB(t) ; ECB(t)=
ECB(t) -E B(t) ;
EB(t)=0
Is EB(t) > LNCL(t)?
Use EB(t) to power
LNCL(t)
EB(t)=EB(t) -L NCL(t) ;
ECB(t)=ECB(t) -L NCL(t) ;
LNCL(t)=0
Yes
Use EB(t) to
power as much of
LNCL(t) as
possible
LNCL(t)=LNCL(t) –
EB(t) ; ECB(t)=
ECB(t) -E B(t) ;
EB(t)=0
Use EPV(t) to charge battery as much
as possible
ECB(t)=ECB(t) + EPV(t);
EB(t)=EB(t)+ E PV(t) ; EPV(t)=0
No
END
Fig. 2. Flowchart forPV-priority1Controller.
G. KumarVenayagamoorthy,R.L.Welch/EngineeringApplicationsofArtificialIntelligence23(2010)249–261 251
ARTICLEINPRESS
This controllerworkswellwhenthereissufficientPVenergy.
However, whenthereisnotsufficientPVenergy,thenthebattery
will notbefullyrechargedandtheloadswillbedropped.The
weather anduserloadsarestochasticinnature;thereforethereis
no onedefinitivemodelatalltimes.Thus,itmakessensetolook
at intelligentmodel-freelearningmethodsofcontrollingsucha
system.
3.2. PV-priority2controller
The secondPV-prioritycontrollerthatisusedinthispaperis
the PV-priority2controller.Asmentionedpreviously,this
controller firstmeetsthecriticalloadthenattemptstocharge
the batteryto70%stateofcharge.Oncethesetwoobjectiveshave
been met,thecontrollerthenattemptstopowerthenon-critical
load usinganyexcessPVenergyorenergyfromthebattery
without depletingitbelow70%stateofcharge,asdetailedin
Fig. 3.
4. ADHDPoptimalcontroller
As previouslymentioned,becauseuserloadsandsolar
insolation arestochasticinnature,itmakessensetolookat
intelligent model-freemethodsofcontrol.Onesuchintelligent
system caninvolvetheuseofadaptivecriticdesigns.ACDsutilize
neural networksandarecapableofoptimizationovertimein
conditions ofnoiseanduncertainty.AfamilyofACDswas
proposed by Werbos (1992) as anewoptimizationtechnique
combining theconceptsofapproximatedynamicprogramming
and reinforcementlearning.WithACDs,foragivenseriesof
control actionsthatmustbetakensequentially(andnotknowing
Use EPV(t) to power L CL (t)
EPV(t)=EPV(t) -L CL(t)
Is EPV(t) > LCL(t)?
Use EPV(t) and EB(t) to power LCL(t)
EB(t)=EB(t) -(L CL(t) -E PV(t)) ;
EPV(t) =0
Yes No
Use EPV to charge battery to 70%
EPV(t)=EPV(t) -(0.7-SOC(t)) ; EB(t)=EB(t)
+(0.7-SOC(t))
Is EPV(t) > (0.7-SOC(t))?
Use EPV to charge battery
EB(t)=EB(t) + EPV(t) ; EPV(t)=0
Yes No
Use any excess EPV(t) to charge battery
Is EB(t)-((LCL(t) + LNCL(t))*anticipated time
until non-zero EPV(t))>70%?
Yes No
Use EPV (t) to power some L NCL(t)
LNCL(t)=LNCL(t) -E PV(t) ; EPV(t)=0;
Use EPV(t) to power LNCL(t)
EPV(t)=EPV(t) -L NCL(t) ; LNCL(t)=0
Is EPV(t) > LNCL(t)?
o N s eY
Use available EB(t) to power remaining LNCL(t), if
possible
EB(t)=EB(t) –L NCL(t);
ECB(t)= ECB(t) –L NCL(t) ; LNCL(t)=0
START
END
to power LCL to power some LNCL(Fig. 3. Flowchart forPV-priority2Controller.
G. KumarVenayagamoorthy,R.L.Welch/EngineeringApplicationsofArtificialIntelligence23(2010)249–261 252
ARTICLEINPRESS
the effectoftheseactionsuntiltheendofthesequence),itis
possible todesignanoptimalcontrollerusingthetraditional
supervised learning-basedneuralnetwork.
The adaptivecriticmethoddeterminesanoptimalcontrolfora
system byadaptingtwoneuralnetworks:an Action network anda
Critic network. TheActionnetworkisresponsiblefordrivingthe
system tothedesiredstates,whiletheCriticnetworkis
responsible forprovidingtheActionnetworkwithperformance
feedback withrespecttoreachingthedesiredstatesovertime.
With thisfeedback,theActionnetworkisabletoadaptits
parameters continuouslytomaximizeitsobjective.TheCritic
network learnstooptimizetheActionnetworkbyapproximating
the Hamilton–Jacobi–Bellmanequationassociatedwithoptimal
control theory.
This Actor–Criticadaptationprocessstartswithanon-optimal
or suboptimalpolicybytheactionnetwork;theCriticnetwork
then guidestheActionnetworktowardanoptimalsolutionat
each successiveadaptation.Duringtheadaptations,neitherofthe
networks needsany‘‘information’’ofanoptimaltrajectory,only
the desiredcostneedstobeknown.Furthermore,thismethod
determines optimalcontrolpolicyfortheentirerangeofinitial
conditions. Additionally,itneedsnoexternaltraining,unlike
other neural-controllers(Venayagamoorthy etal.,2002).
The designladderofACDsincludesthreebasicimplementa-
tions: HeuristicDynamicProgramming(HDP),DualHeuristic
Programming (DHP)andGlobalizedDualHeuristicProgramming
(GDHP), intheorderofincreasingpowerandcomplexity.The
interrelationships betweenmembersoftheACDfamilyhavebeen
generalized andexplainedin(Prokhorov andWunsch,1997). In
this paper,anAction-dependentHDP(ADHDP)approachis
adopted forthedesignofanoptimalPVcontroller.Action-
dependent adaptivecriticdesignsdonotneedsystemmodelsto
develop theoptimalcontrolpolicy(actionnetworkoutput).
As mentioned,theobjectiveoftheoptimalPVcontrolis
threefold –tomaximizeorfullydispatchtherequiredenergyto
the criticalloadsatalltimes,dispatchenergytochargethe
battery whenevernecessarysoastodispatchenergytothecritical
loads intheabsenceofenergyfromthecollectorandthelast
objective istodispatchenergytothenon-criticalloadsnot
comprising onthefirsttwoobjectives.Theoptimalcontrolleris
not usedforinstanceswherethereissufficientsolarenergyto
power allloadsaswellascompletelychargethebattery.When
this occurs,allloadsaresatisfiedandthebatteryiscompletely
charged.
This optimalcontrollerusestwonetworks(theActionand
Critic networks)aspreviouslymentioned.Theinputstothe
Action networkcorrespondtothestatesofthesystemwhilethe
outputs correspondtotheamountofenergytobedispatchedto
the criticalloads,batteryandnon-criticalloads.Theinputstothe
Critic consistoftheinputstotheActionnetworkattime t, t1
and t2, aswellastheoutputsoftheActionnetworkattime t,
t1 and t2. TheCriticthenusestheinformationfromthe
current statesandactionsinthecurrenttimestep(aswellasfrom
the recentpast)toderivetheActionnetworkovertimetoevolve
an optimalcontrolpolicy. Fig. 4 shows theconnectionbetween
the Actionnetwork,CriticnetworkandthePVsystem.