Files
AlphaFlow-MT5-ML-DL-Trading…/features/__pycache__/feature_engineering.cpython-311.pyc
T

202 lines
30 KiB
Plaintext
Raw Normal View History

2025-03-02 22:25:33 +01:00
§
2026-03-11 18:34:58 +00:00
*µ±i—Wã óUddlmZddlZddlZddlmZmZmZmZddl Z
ddl Z ddl
Z
ddlmZddlmZddlmZmZddlmZdd d d
d dgd ¢hd
£dœZded<drdZdrdZdrdZdgd¢fdsdZdrd „Zdtd!„Zdud"„Zdvd%„Z dwdxd(„Z!dvd)„Z"dwdxd*„Z#dyd.„Z$dzd{d1„Z%d|d4„Z&dzd}d5„Z'd~dd:„Z(d€dd=„Z) ddƒdD„Z* d„d…dE„Z+ed†dK„¦«Z, d‡dˆdM„Z-d‰dŠdO„Z.ddŒdR„Z/ddW„Z0dŽd\„Z1dedfddb„Z2 dd}dc„Z3ddf„Z4 dd“di„Z5 d”d•dp„Z6 dd—dq„Z7dS)˜é)Ú annotationsN)ÚDictÚListÚOptionalÚTuple)Únjit)Úfft)ÚadfullerÚkpss)ÚStandardScalerTçš™™™™™©?Fé)Ú
pct_changeÚdiff1Ú log_diff1Ú
seasonal_diff> ÚlowÚhighÚopenÚcloseÚvolumeÚ tick_volumeÚstationary_flagÚrolling_adf_pvalÚrolling_adf_stat)ÚenabledÚ adf_alphaÚ
2025-03-02 22:25:33 +01:00
kpss_alphaÚ
keep_originalÚmax_diffÚseasonal_periodÚtransform_orderÚ exclude_colsrÚSTATIONARITY_CFGÚdfú pd.DataFrameÚreturnc ó<tj|dddddd¬¦«}|S)z•
2026-03-11 18:34:58 +00:00
Adds a wide range of technical analysis indicators using the 'ta' library.
Modifies the DataFrame in place and returns it for chaining.
rrrrrT)rrrrrÚfillna)ÚtaÚadd_all_ta_features©r%s ú$/app/features/feature_engineering.pyr+r+'s7õ
Ô Ø
Ø
Ø
2025-03-02 22:25:33 +01:00
Ø ØØØð
2026-03-11 18:34:58 +00:00
ñ
ô
€Bð €Iócóh|d d¬¦« ¦«|d<|S)z:Example custom feature: a rolling mean of the close price.ré
©ÚwindowÚrolling_mean_10)ÚrollingÚmeanr,s r-Úcreate_custom_featurer68s3à˜wœK×/°rÐA€BÐÑØ
€Ir.cóV| ¦«}|d|dz
|d<|S)zSpread between high and low.rrÚspread)Úcopy)r%Údfcs r-r8r8As*à
2025-03-02 22:25:33 +01:00
'Š'‰)Œ)€Cؘ”K # e¤*Ñ,€Cˆ €Jr.é2)ééér0ÚcolÚstrÚintÚlagsú List[int]có°| ¦«}|D]=Š|| ||¬¦« ˆfdd¬¦«|d<Œ>|S)z*Rolling autocorrelation for multiple lags.)r2Ú min_periodscó0| ¬¦«S)N)Úlag)Úautocorr)ÚxrHs €r-ú<lambda>z!auto_corr_multi.<locals>.<lambda>Qsø€˜QŸZšZ¨C˜0€r.F)ÚrawÚ autocorr_)r9r4Úapply)r%r?rArCr:rHs @r-Úauto_corr_multirOHsqø€ð
2026-03-11 18:34:58 +00:00
'Š'‰)Œ)€CØð
ð
ˆà ŒHß
ŠW˜A¨1ˆ
ŠUÐ0°eˆUÑ
 Ð ˜Ð Ð ÑÐð
€Jr.cóÆ| ¦«}|d|dk t¦«|d<|d|dz
 dtj¦«}|d|dz
 ¦«|dzz |d<|d|dz
 ¦«|d ¦«dzz |d <|S)
Adds candle-specific features:
- candle_way (1 if close > open else 0)
- fill (real body / range)
- amplitude (abs(close - open) / open)
rrÚ
2025-03-02 22:25:33 +01:00
candle_wayrrrgñh㈵øä>ÚfillÚ amplitude)r9ÚastyperBÚreplaceÚnpÚnanÚabs)r%r:Úrngs r-Úcandle_informationrZVð
2026-03-11 18:34:58 +00:00
'Š'‰)Œ)€Cؘ¨¨F¬ Ò;½CÑ@€Cˆ ÑØ ˆvŒ;˜˜UœÑ
2025-03-02 22:25:33 +01:00
,¨Qµ´Ñ
2026-03-11 18:34:58 +00:00
7€CØw”< # f¤+Ñ-×4¸¸d¹
ÑC€Cˆ˜Gœ  s¨6¤{Ñ9¸¼[¿_º_Ñ=NÔ=NÐQUÑ=UÑV€Cˆ ÑØ €Jr.cóÞ| ¦«}tj|| d¬¦«¦«|d|<|d| |¦«|d|<|S)zR
Create log(price) and n-period log-return: log_ret_n = log(col).diff(n).
gê-™—q=©ÚlowerÚlog_Úlog_ret_)r9rVÚlogÚclipÚdiff)r%r?rAr:s r-Ú
log_transformrceskð
'Š'‰)Œ)€C團s 3œxŸ}š}°5˜}Ñ:€Cˆ ˆsˆ ˆ ÑØ˜l S˜l˜lÔÑ3€CˆˆÑØ €Jr.có¢| ¦«}|| ¦«|d<|d ¦«|d<|S)z-Velocity and acceleration for a given column.ÚvelocityÚ acceleration)r9rb)r%r?r:s r-Úmathematical_derivativesrgpsCà
'Š'‰)Œ)€Cؘ#”h—mmo”o€Cˆ
˜jœ/×0€CˆÑØ €Jr.r2Úfloatcót|¦«}|dkr tjStjtj|d|dz ¦«dz¦«}t j|dt jd¦«z|zz ¦«S)Nr<rrré)ÚlenrVrWÚsumr`ÚmathÚsqrt)r2rAÚsum_sqs r-Úparkinson_estimatorrp{snÝ ˆF‰ Œ €A؈1‚u€uÝŒvˆ
Ý
ŒV•B”F˜6 &œ>¨F°5¬MÑ:¸
@€FÝ Œ9V˜q¥4¤8¨A¡;¤;™°Ñ 4r.éÚ window_sizecó| ¦«}tjd|j¬¦«}t |t |¦«¦«D]+}|j||z
|}t|¦«|j|<Œ,||d<|S)float64©ÚdtypeÚindexÚrolling_volatility_parkinson)r9ÚpdÚSeriesrwÚrangerkÚilocrp©r%rrr:Ú rolling_volÚws r-Úmoving_parkinson_estimatorrƒsØ
'Š'‰)Œ)€CÝ”) )°3´=€KÝ
 ¤Ñ
Ø ŒH‘_ qÐ Ý1°!Ñ Ô˜ÑÐØ*5€CÐ €Jr.có.t|¦«}|dkr tjStj|d|dz ¦«dz}tj|d|dz ¦«dz}t jtj||z¦«¦«S)Nr<rrrrr)rkrVrWr`rmrnr5)r2rAÚterm1Úterm2s r-Úyang_zhang_estimatorr…s}Ý ˆF‰ Œ €A؈1‚u€uÝŒvˆ
Ý ŒF6˜&”> F¨5¤MÑ 2°aÑ 7€EÝ ŒF6˜'”? V¨F¤^Ñ 4¸Ñ 9€EÝ Œ9•R”W˜U U™]Ñ ,r.có| ¦«}tjd|j¬¦«}t |t |¦«¦«D]+}|j||z
|}t|¦«|j|<Œ,||d<|S)NrtruÚrolling_volatility_yang_zhang)r9ryrzrwr{rkr|r…r}s r-Úmoving_yang_zhang_estimatorrˆsØ
'Š'‰)Œ)€CÝ”) )°3´=€KÝ
 ¤Ñ
Ø ŒH‘_ qÐ Ý2°1Ñ Ô˜ÑÐØ+6€CÐ €Jr.ÚPextÚ thresholdcó8||z
2025-03-02 22:25:33 +01:00
|z }||krdS|| krdSdS)Nr<éÿÿÿÿr©)r‰rÚvars r-Údc_eventr£s7Ø ˆt‰8
2026-03-11 18:34:58 +00:00
€CØ
ˆÐ؈
ˆyˆjÒÐØˆrØ ˆ1r.ç{®Gáz„?úTuple[List[int], List[int]]có”| ¦«}|dj}gg}}|d}d}tdt|¦«¦«D]x}||} t | ||¦«}
|
dkr| |¦«d}| }Œ;|
dkr| |¦«d}| }Œ[|dkr | |kr| }Œj|dkr| |kr| }Œy||fS)Nrrr<r)r9Úvaluesr{rkrÚappend) r%rr:ÚpricesÚdc_upÚdc_downrŠÚ directionrr‰Úflags r-Ú calculate_dcr¬Ø
'Š'‰)Œ)€CØ
Œ\Ô
€Fؘˆ7€EØ !Œ9€DØ€IÝ
1•c˜&k”kÑ
ðˆØ 1ŒIˆÝ˜˜4 ÑØ 1Š9ˆ9Ø LŠL˜‰OŒOˆO؈I؈DˆDØ
RŠZˆZØ NŠN˜1Ñ Ô Ð ØˆI؈DˆDà˜AŠ~ˆ~ ! d¢( (ØØ˜b’ Q¨¢X øØ 'ˆr.Ú dc_events_upÚdc_events_downcóztt|¦«¦«}tt|¦«¦«}||fS)N)ÚlistÚsorted)rr%Útrend_events_downÚtrend_events_ups r-Úcalculate_trendr£Ås8ÝV NÑÝ6 ,Ñ0€OØ ˜ -r.có*| ¦«}t||¬¦«\}}t|||¦«\}}tj|d<d|j|df<d|j|df<|d ¦« ¦«|d<|S)N)rÚ
market_regimer<r)r9rrVrWÚlocÚffillÚbfill)r%rr:r—r˜Út_downÚt_ups r-Úmarket_regime_dcr«Ës”Ø
'Š'‰)Œ)€CÝ! Ð;N€Eˆ" 5¨'°3Ñ7L€Fˆœ6€CˆÑØ%&€C„GˆD "Ø'(€C„GˆF ˜Ô?€CˆÑØ €Jr.rr0Ún1Ún2cóH| ¦«}|| |d¬¦« ¦«}|| |d¬¦« ¦«}||z
|d<|ddk t¦«|d<|S)NF©ÚspanÚadjustÚ kama_diffrÚ
kama_trend)r9Úewmr5rTrB)r%r?r­r:Ú
short_kamaÚ long_kamas r-Úkama_market_regimer·ÖsØ
'Š'‰)Œ)€CØS” 2¨eÑ;€JØC”  "¨U Ñ3×:€IØ! IÑ-€Cˆ ÑØ˜[Ô)¨QÒ6µsÑ;€Cˆ ÑØ €Jr.r<Úlookbackcóz| ¦«}gd¢}|D]}tj||<Œt|t |¦«¦«D]ð}|dj||z
}|dj||z
}|dj|}|dj|} | |krH||j|j|df<| |j|j|df<| |z
|j|j|df<||krH||j|j|df<||j|j|df<||z
|j|j|d f<Ζ|S)
N)ÚBullish_gap_infÚBullish_gap_supÚBullish_gap_sizeÚBearish_gap_infÚBearish_gap_supÚBearish_gap_sizerrr¿)r9rVrWr{rkr|Úatrw)
r%r¸r:ÚcolsÚcrÚ prev_highÚprev_lowÚ curr_highÚcurr_lows
r-Ú
gap_detectionrÇâs_Ø
'Š'‰)Œ)€Cð ð ð €DðððˆÝˆˆA‰ˆÝ
8S ™XœXÑ
Lð LˆØ˜”KÔ$ Q¨¡\Ô2ˆ Øu”:”? 1 x¡<Ô0ˆØ˜”KÔ$ 'ˆ Øu”:”? %ˆØ  Ð Ø6?ˆCŒF3”9˜Q”<Ð!2Ð 3Ø6>ˆCŒF3”9˜Q”<Ð!2Ð 3Ø7?À)Ñ7KˆCŒF3”9˜Q”<Ð!3Ð  Ð Ø6?ˆCŒF3”9˜Q”<Ð!2Ð 3Ø6>ˆCŒF3”9˜Q”<Ð!2Ð 3Ø7?À)Ñ7KˆCŒF3”9˜Q”<Ð!3Ð 4øØ €Jr.ÚstandardçÚ
type_rangeÚstrenghtÚperiodcó¢| ¦«}|dkr'|d|dz
 ¦«|d<n<|dkr'|d|dz
 ¦«|d<ntd¦«|d|dz d z
 ¦«|d
<|d |¦« ¦«|d <d |d
<|d||d zk}d |j|d
f<|d
|d
 d ¦« d ¦«z t¦«|d<|S)NrÈrrÚ candle_rangeÚextremrrz1Invalid 'type_range'. Use 'standard' or 'extrem'.r<Ú VariationÚSTDrÚ displacementÚred_displacement)
r9rXÚ
ValueErrorr4Ústdr¦Úshiftr)rTrB)r%r:Úmasks r-Údisplacement_detectionrÙþsPð
'Š'‰)Œ)€CØÐØ" 7œ|¨c°&¬kÑ9×@ˆˆÐØ Ð Ø" 6œ{¨S°¬ZÑ7×>ˆˆÐåИGœ  s¨6¤{Ñ2°QÑ=€Cˆ ÑØ,¨VÑ:€Cˆ€CˆÑØ ˆ~Ô  ¨C°¬JÑ!6Ò 6€DØ$%€C„GˆD Ñ" 2°S¸Ô5H×5NÒ5NÈqÑ5QÔ5Q×5XÒ5XÐYZÑ5[Ô5[Ñ[×cÕdgÑh€CÐÑØ €Jr.có”| ¦«}tjd|j¬¦«}tjd|j¬¦«}tjd|j¬¦«}t |t |¦«¦«D]µ}||j||z
|j} t| d¬¦«}
|
d|j|<|
d|j|<|
d|krdnd|j|<Œj#t$r?tj |j|<tj |j|<tj |j|<YŒ²wxYw||d<||d<||d <|S)
zBCompute rolling ADF p-values and a stationarity flag (diagnostic).rtruÚAIC©Úautolagrr<rrr) r9ryrzrwr{rkr|r”r
Ú ExceptionrVrW) r%r?rrÚp_value_thresholdr:Úadf_statÚadf_pvalršrÚ
slice_dataÚresults r-Úrolling_adf_with_flagräsUð
'Š'‰)Œ)€CÝŒy˜´ Ð:€HÝŒy˜y°´ Ð:€HÝ
Œ9˜9¨C¬IÐ 6€Då
 ¤Ñ

"ˆØ˜”X”] 1 {¡?°QÐ#6Ô>ˆ
ð˜j°%Ð8ˆ% aœyˆHŒM˜ Ø% aœyˆHŒM˜ Ø & q¤ Ð,=Ò =Ð =˜1˜1À1ˆDŒIa‰LˆLøÝð!œvˆHŒM˜ Ý!œvˆHŒM˜!Ñ Ýœ6ˆDŒIa‰LˆLˆ "øøøð
'€CÐÑØ&€CÐÑØ!€CÐÑØ €JsÂ#A C-Ã-AD6Ä5D6Úclosesú
np.ndarrayÚupÚdownÚhorizoncóTt|¦«}tj|tj¦«}t |¦«D]i}||}|d|zz}|d|z
z} t ||z|¦«}
t |dz|
¦«D](} || |krd||<n|| | krd||<nŒ)Œj|S)Nr<r)rkrVÚfullrWr{Úmin) rAÚlabelsrÚcurrentÚupperr]ÚendÚjs r-Ú_compute_double_barrier_labelsrò3å ˆF‰ Œ €AÝ
ŒWQœÑ
Ô
€FÝ
1‰XŒXð ð ˆØ˜”)ˆØ˜1˜r™6ÑØ˜1˜t™8Ñ$ˆÝ!g‘+˜qÑ!ˆÝq˜1u˜ ð ˆaŒy˜EÒq‘ ØØaŒy˜EÒq Øð"øð €Mr.ç{®Gázt?có¦| ¦«}|dj}t||||¦«}||d<| dgd¬¦«|S)NrÚ
barrier_labelT©ÚsubsetÚinplace)r9r”Údropna)r%r:s r-Úset_double_barrier_labelrúEsXð
'Š'‰)Œ)€CØ
Œ\Ô
€Få
+¨F°B¸¸
F€Fà!€CˆÑ؇J‚JЀJÑ €Jr.縅ëQ¸ž?có,| ¦«}|d | ¦«|dz dz
|d<tj|d<d|j|d|kdf<d|j|d| kdf<| dgd¬¦«|S) Nrgð?Ú
future_returnÚfuture_market_regimer<rTrö)r9r×rVrW)r%rr:s r-Úfuture_DC_market_regimerÿUØ
'Š'‰)Œ)€Cؘwœ<×-¨w¨hÑ7¸#¸g¼,ÑÑL€CˆÑÝ"$¤&€CÐÑØIJ€C„GˆCÔ  IÒ -Ð/EÐ FØJK€C„GˆCÔ  Y  .Ð0FÐ ‡J‚JЀJÑ €Jr.r>Ú n_componentscó | ¦«}tjt||j¦«¦«}t d|dz¦«D]/}|t
|¦«kr||n tj|d|<Œ0|S)z
Global FFT magnitudes (same values on all rows). For truly time-local
frequency content, implement a rolling FFT (heavier) instead.
r<Ú fft_comp_)r9rVrXr r”r{rkrW)r%r?rr:Úfft_valsrs r-Úadd_fourier_featuresrbs€ð
'Š'‰)Œ)€CÝŒv•c˜#˜cœ(œ/Ñ+€HÝ
1l 
LðLˆØ./µ#°h±-´-Ò.?Ð.?˜x¨œ{˜{ÅRÄVˆˆO˜ˆOˆOÑÐØ €Jr.Úseriesú pd.Seriesrrcó| ¦« t¦«}t|¦«dkrtjtjddœS t
|d¬¦«d}n#t$rtj}YnwxYw t|dd¬ ¦«d}n#t$rtj}YnwxYwt j |¦« o||k}t j |¦« o||k}|||o|dœS)
NrqF)Úadf_pÚkpss_pÚ
stationaryrÛr<Úauto)Ú
regressionÚnlags)
rTrhrkrVrWr
r Úisnan)rrrÚsrr Úok_adfÚok_kpsss r-Ú_is_stationaryrqsØ
Š
Œ×Ò%€AÝ
ˆ1v„v‚{€{Ýœ­2¬6ÀÐݘ EÐ*¨1ÔˆøÝ ððð݈ˆˆðøøøðÝa C¨vÐ6°qÔˆøÝ ððð݈ˆˆðøøøå”(˜5‘/”/Ð
° Ò(9€FÝ”8˜>¨&°:Ò*=€GØ f¸VÐ=OÈÐ Qs$ÁA3Á3B  B ÂB)Â)CÃCrÚkindr!ú
Optional[int]có8|dkr| ¦«S|dkr| d¦«S|dkr;tj| d¬¦«¦« d¦«S|dkr|r|dkr| |¦«S|S)Nrrr<rrr\r)rrbrVÚlog1pra)rrr!s r-Ú_apply_transformrØ ˆ|ÒÐØ|Š|‰~Œ~ÐØ ˆw‚ØvŠva‰yŒyÐØ ˆ{ÒÐÝŒx˜Ÿš Q˜œÑ-¨aÑ ˆÒР?аÈÒ7JÐ7JØvŠv €Hr.úOptional[List[str]]ÚcfgÚexcludeúTuple[pd.DataFrame, Dict]c
óÈ| dd¦«s|ddifS| ¦«}|idœ}|€8| tjg¬¦«j ¦«}t| dt¦«¦«¦«Š|r |¦«ˆfd|D¦«}gg}}|D]Í}||} t| |d |d
¦«}
|
d|d œ} |
d r | |d
|<ŒBd} |dD]}
t| |
| d¦«¦«}t||d |d
¦«}|d rP|d|
}|||<d|
i|¥| d<|| d<|  |¦«|ds|  |¦«d} nŒœ| sØ|  ¦«}tdt| dd¦«¦«dz¦«D]}| d¦«}t||d |d
¦«}|d rS|d|}|||<dd|i|¥| d<|| d<|  |¦«|ds|  |¦«d} nŒ| |d
|<ŒÏ|r0| t!t|¦«¦«¬¦«}| ¦«}||fS)
For each numeric feature, test ADF+KPSS. If non-stationary,
apply transforms in cfg['transform_order'] (no look-ahead), re-test,
and keep only features that pass. Returns (df_out, report).
rTF)ÚconfigÚfeaturesN©Úincluder#cóg|]}|v¯|Œ S)Ú.0rÂÚ exclude_sets €r-ú
<listcomp>z.ensure_stationary_features.<locals>.<listcomp>¦s#ø€Ð 4!˜q¨ Ð3ˆ3r.rr)ÚoriginalÚappliedÚ final_colr
rr"r!Ú__Ú transformr&r'rr<r rÚ__diffrb)Úcolumns)Úgetr9Ú
select_dtypesrVÚnumberr+ÚtolistÚsetÚupdaterrr•r{rBrbÚdroprŸ)r%rrÚdf_outÚreportÚ drop_colsÚ
added_colsr?rÚbaseÚentryr&rÚs_tÚtest_tÚnew_colÚs_fÚtest_fr#s @r-Úensure_stationary_featuresr?Žsoø€ð 7Š79˜ I˜
WŠW‰YŒY€FØ!¨rÐ2€Fð €|Ø×#­R¬Y¨KÐåc—g’g˜n­c©e¬eÑ5€KØð×Ò˜7Ñ 4 4€Dà ˆz€Iàð**Ø 3ŒKˆÝ˜a  [Ô!1°3°|Ô3DÑØ!¨dÀÐà  Ô ð Ø&+ˆF ˜sÑ ðˆØÐ ð ˆ" 1 d¨C¯GªGÐ4EÑ,FÔ,FÑGˆ# ¨[Ô)9¸3¸|Ô;LÑMˆ
Ø Ð* *Ø"%wØ$/°Ð#@¸Ð#@ Ø%,×! ˜×$ SÑØð
ðð Ø—&&(”(ˆ˜1c #§'¢'¨*°aÑ"8Ô"8Ñ9¸
ð
Ø—hh˜q‘k”kݨS°Ô-=¸sÀ<Ô?PÑQؘ Ø!$Ð/¨AÐ/GØ&)F˜7OØ(3°Z¸A°Z°ZÐ'JÀ6Ð'JE˜$Ø)0E˜×% ˜ÔÑ" ð#(ˆˆzÔ˜3ÑÑàð¥T­#¨i©.¬.Ñ%9Ô%9Ñð]Š]‰_Œ_€Fà 6ˆr.cójt ¦«}||d<t||¬¦«\}}|S)z
Legacy simple ADF-only differencing (kept for backward compatibility).
Prefer `ensure_stationary_features` for robust ADF+KPSS handling.
r©r)r$r9r?)r%rrr3Ú_s r-Úapply_stationarity_testrCàs:õ ×
Ò