Python 从Pandas到Statsmodels的OLS中不推荐的滚动窗口选项
正如标题所示,Pandas中ols命令中的滚动功能选项在哪里迁移到statsmodels中?我好像找不到它。 熊猫告诉我厄运正在酝酿中:Python 从Pandas到Statsmodels的OLS中不推荐的滚动窗口选项,python,pandas,deprecated,statsmodels,Python,Pandas,Deprecated,Statsmodels,正如标题所示,Pandas中ols命令中的滚动功能选项在哪里迁移到statsmodels中?我好像找不到它。 熊猫告诉我厄运正在酝酿中: FutureWarning: The pandas.stats.ols module is deprecated and will be removed in a future version. We refer to external packages like statsmodels, see some examples here: http://stat
FutureWarning: The pandas.stats.ols module is deprecated and will be removed in a future version. We refer to external packages like statsmodels, see some examples here: http://statsmodels.sourceforge.net/stable/regression.html
model = pd.ols(y=series_1, x=mmmm, window=50)
事实上,如果你做了如下事情:
import statsmodels.api as sm
model = sm.OLS(series_1, mmmm, window=50).fit()
print(model.summary())
您会得到结果(窗口不会影响代码的运行),但您只会得到整个周期内回归运行的参数,而不是它应该工作的每个滚动周期的一系列参数。使用sklearn滚动测试版
import pandas as pd
from sklearn import linear_model
def rolling_beta(X, y, idx, window=255):
assert len(X)==len(y)
out_dates = []
out_beta = []
model_ols = linear_model.LinearRegression()
for iStart in range(0, len(X)-window):
iEnd = iStart+window
model_ols.fit(X[iStart:iEnd], y[iStart:iEnd])
#store output
out_dates.append(idx[iEnd])
out_beta.append(model_ols.coef_[0][0])
return pd.DataFrame({'beta':out_beta}, index=out_dates)
df_beta = rolling_beta(df_rtn_stocks['NDX'].values.reshape(-1, 1), df_rtn_stocks['CRM'].values.reshape(-1, 1), df_rtn_stocks.index.values, 255)
为完整性增加了一个更快的
numpy
唯一的解决方案,该解决方案将计算仅限于回归系数和最终估计
Numpy滚动回归函数
import numpy as np
def rolling_regression(y, x, window=60):
"""
y and x must be pandas.Series
"""
# === Clean-up ============================================================
x = x.dropna()
y = y.dropna()
# === Trim acc to shortest ================================================
if x.index.size > y.index.size:
x = x[y.index]
else:
y = y[x.index]
# === Verify enough space =================================================
if x.index.size < window:
return None
else:
# === Add a constant if needed ========================================
X = x.to_frame()
X['c'] = 1
# === Loop... this can be improved ====================================
estimate_data = []
for i in range(window, x.index.size+1):
X_slice = X.values[i-window:i,:] # always index in np as opposed to pandas, much faster
y_slice = y.values[i-window:i]
coeff = np.dot(np.dot(np.linalg.inv(np.dot(X_slice.T, X_slice)), X_slice.T), y_slice)
estimate_data.append(coeff[0] * x.values[window-1] + coeff[1])
# === Assemble ========================================================
estimate = pandas.Series(data=estimate_data, index=x.index[window-1:])
return estimate
将numpy导入为np
def rolling_回归(y,x,窗口=60):
"""
y和x必须是熊猫系列
"""
#==清理============================================================
x=x.dropna()
y=y.dropna()
#==根据最短距离进行修剪================================================
如果x.index.size>y.index.size:
x=x[y.指数]
其他:
y=y[x.索引]
#==验证是否有足够的空间=================================================
如果x.index.size<窗口:
一无所获
其他:
#==如果需要,添加一个常量========================================
X=X.至_帧()
X['c']=1
#==循环。。。这是可以改进的====================================
估计数据=[]
对于范围内的i(窗口,x.index.size+1):
X_slice=X.values[i-window:i,:]#始终以np索引,而不是熊猫索引,速度更快
y_slice=y.values[i-window:i]
系数=np.dot(np.dot(np.linalg.inv(np.dot(X_slice.T,X_slice)),X_slice.T,y_slice)
估算数据追加(系数[0]*x.values[window-1]+coeff[1])
#==组装========================================================
估算=熊猫。系列(数据=估算\数据,索引=x.index[window-1:])
收益估计
注释 在某些特定情况下,只需要最终估计回归,
x.rolling(window=60)。apply(my_ols)
似乎有点慢
作为提醒,回归系数可以作为矩阵积计算,如您所读。与使用
statsmodels
中的ols相比,这种通过numpy
的矩阵乘法的方法可以在一定程度上加快该过程。该产品以coeff=…
开头的行表示,我创建了一个ols
模块,用于模拟熊猫不推荐的移动工具
;是的
它有三个核心类:
:静态(单窗口)普通最小二乘回归。输出是NumPy数组OLS
:滚动(多窗口)普通最小二乘回归。输出是高维NumPy数组RollingOLS
:将PandasRollingOLS
的结果包装在pandas系列和数据帧中。旨在模仿弃用熊猫模块的外观RollingOLS
RegressionResultsWrapper
例如:
import urllib.parse
import pandas as pd
from pyfinance.ols import PandasRollingOLS
# You can also do this with pandas-datareader; here's the hard way
url = "https://fred.stlouisfed.org/graph/fredgraph.csv"
syms = {
"TWEXBMTH" : "usd",
"T10Y2YM" : "term_spread",
"GOLDAMGBD228NLBM" : "gold",
}
params = {
"fq": "Monthly,Monthly,Monthly",
"id": ",".join(syms.keys()),
"cosd": "2000-01-01",
"coed": "2019-02-01",
}
data = pd.read_csv(
url + "?" + urllib.parse.urlencode(params, safe=","),
na_values={"."},
parse_dates=["DATE"],
index_col=0
).pct_change().dropna().rename(columns=syms)
print(data.head())
# usd term_spread gold
# DATE
# 2000-02-01 0.012580 -1.409091 0.057152
# 2000-03-01 -0.000113 2.000000 -0.047034
# 2000-04-01 0.005634 0.518519 -0.023520
# 2000-05-01 0.022017 -0.097561 -0.016675
# 2000-06-01 -0.010116 0.027027 0.036599
y = data.usd
x = data.drop('usd', axis=1)
window = 12 # months
model = PandasRollingOLS(y=y, x=x, window=window)
print(model.beta.head()) # Coefficients excluding the intercept
# term_spread gold
# DATE
# 2001-01-01 0.000033 -0.054261
# 2001-02-01 0.000277 -0.188556
# 2001-03-01 0.002432 -0.294865
# 2001-04-01 0.002796 -0.334880
# 2001-05-01 0.002448 -0.241902
print(model.fstat.head())
# DATE
# 2001-01-01 0.136991
# 2001-02-01 1.233794
# 2001-03-01 3.053000
# 2001-04-01 3.997486
# 2001-05-01 3.855118
# Name: fstat, dtype: float64
print(model.rsq.head()) # R-squared
# DATE
# 2001-01-01 0.029543
# 2001-02-01 0.215179
# 2001-03-01 0.404210
# 2001-04-01 0.470432
# 2001-05-01 0.461408
# Name: rsq, dtype: float64
对于一列中的滚动趋势,可以使用:
import numpy as np
def calc_trend(window:int = 30):
df['trend'] = df.rolling(window = window)['column_name'].apply(lambda x: np.polyfit(np.array(range(0,window)), x, 1)[0], raw=True)
然而,在我的例子中,我浪费了时间去寻找关于日期的趋势,日期在另一列中。我必须手动创建功能,但这很容易。首先,将TimeDate转换为int64,表示从t_0开始的天数:
xdays = (df['Date'].values.astype('int64') - df['Date'][0].value) / (1e9*86400)
然后:
def calc_趋势(窗口:int=30):
对于范围内的t(len(df)):
如果t
@CharlesPlager感谢您提醒我注意这一点,链接已更新。您的示例代码在Python 3.6.1上无法运行。请参阅@SamArthurGillam我已更新为一个工作示例。引发异常是因为datareader
没有正确地拉入每月频率。(看起来像是一个fq
param被添加到圣路易斯联邦网址下载。)这反过来意味着其中一列是一堆零,这将导致一个奇异矩阵并将事情搞砸。嗨,布拉德,我发现你的示例数据有点恼人。只是想说明一下:您的模型使用术语价差和黄金变化作为贸易加权美元价值变化的解释变量?价差变化,或者x的第一列是无风险回报,黄金改变市场回报?你为什么不使用更简单的变量,比如y股票的变化,t利率(不是变化)作为无风险回报,sp500的变化作为市场回报呢?@LucaReichelt这是一个公认的人为的例子,但将某人免费提供给你的答案称为“恼人”有点过分。欢迎您直接建议对答案进行编辑
def calc_trend(window:int=30):
for t in range(len(df)):
if t < window//2:
continue
i0 = t - window//2 # Start window
i1 = i0 + window # End window
xvec = xdays[i0:i1]
yvec = df['column_name'][i0:i1].values
df.loc[t,('trend')] = np.polyfit(xvec, yvec, 1)[0]