Reference: https://github.com/pandas-dev/pandas/issues/17127
Code
def persistence(self):
self.pd.to_csv(order_record_file_name, encoding='utf-8')
def _update_order_record(self, order_detail):
order_pd_index = "%d" % order_detail.orderId
if order_pd_index in self.pd.index.values:
logging.info("Received order_detail:%s", order_detail.to_json())
# of couse it runs here and i get he above logging, and then i update the cell value
self.pd.at[order_pd_index, 'tradePrice'] = order_detail.tradePrice
self.pd.at[order_pd_index, 'tradedVolume'] = order_detail.tradedVolume
self.pd.at[order_pd_index, 'tradedAmount'] = order_detail.tradedAmount
self.pd.at[order_pd_index, 'orderStatus'] = order_detail.orderStatus
# here write it back to file
self.persistence()
# my temp solution is to check whether the recorded value is the same as input
for key in ['tradePrice', 'tradedVolume', 'tradedAmount', 'orderStatus']:
if self.pd.loc[order_pd_index][key] != order_detail.__dict__[key]:
logging.warn("%s not the same as record file" % key)
self.pd.at[order_pd_index, key] = order_detail.__dict__[key]
self.persistence()
else:
logging.info("key %s after updated is %s" % (key, str(self.pd.loc[order_pd_index][key])))
Logs
Round 1 good case
at 2017-07-26 09:21:00,913
2017-07-26 09:21:00,913 139936711816960 StrategyOrderManagement.py,line 107 INFO: Received order_detail:{ "orderId": 1707260921000002148, "tradedAmount": 0.0, "userId": 95, "tradedVolume": 0, "orderStatus": 15, "strategyId": 40, "volumeOriginal": 11, "tradingDay": "20170726", "positionEffect": 0, "exchangeOrderId": 0, "createTime": 1501032060815, "hedgeFlag": 51, "orderPrice": 14910.0, "tradePrice": 0.0, "instrument": "CZCE.CF709", "bsFlag": -1}
2017-07-26 09:21:00,917 139936711816960 StrategyOrderManagement.py,line 123 INFO: key tradePrice after updated is 0.0
2017-07-26 09:21:00,918 139936711816960 StrategyOrderManagement.py,line 123 INFO: key tradedVolume after updated is 0.0
2017-07-26 09:21:00,919 139936711816960 StrategyOrderManagement.py,line 123 INFO: key tradedAmount after updated is 0.0
2017-07-26 09:21:00,920 139936711816960 StrategyOrderManagement.py,line 123 INFO: key orderStatus after updated is 15.0
Round 2 error case
at 2017-07-26 09:21:02,704
2017-07-26 09:21:02,704 139936711816960 StrategyOrderManagement.py,line 107 INFO: Received order_detail:{ "orderId": 1707260921000002148, "tradedAmount": 0.0, "userId": 95, "tradedVolume": 11, "orderStatus": 19, "strategyId": 40, "volumeOriginal": 11, "tradingDay": "20170726", "positionEffect": 0, "exchangeOrderId": "2017072601552578", "createTime": 1501032060815, "hedgeFlag": 51, "orderPrice": 14910.0, "tradePrice": 14910.0, "instrument": "CZCE.CF709", "bsFlag": -1}
2017-07-26 09:21:02,707 139936711816960 StrategyOrderManagement.py,line 123 INFO: key tradePrice after updated is 14910.0
2017-07-26 09:21:02,707 139936711816960 StrategyOrderManagement.py,line 119 **WARNING: user dh515-02-1d strategy sz-strategy02-1d critial error:tradedVolume not the same as record file**
2017-07-26 09:21:03,858 139936711816960 StrategyOrderManagement.py,line 123 INFO: key tradedAmount after updated is 0.0
2017-07-26 09:21:03,859 139936711816960 StrategyOrderManagement.py,line 123 INFO: key orderStatus after updated is 19.0
Problem description
Everyday my program does the same operation more than 1000 times, but failed less than 1 time per several days, and maybe no problem in months... Sometimes on the field tradedVolume, sometimes the other filed such as orderStatus.
Output of pd.show_versions()
>>> pd.show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 2.7.5.final.0
python-bits: 64
OS: Linux
OS-release: 3.10.0-123.9.3.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: zh_CN.UTF-8
pandas: 0.18.1
nose: None
pip: 7.1.0
setuptools: 0.9.8
Cython: None
numpy: 1.11.1
scipy: None
statsmodels: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.5.3
pytz: 2016.6.1
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
boto: None
pandas_datareader: None
<details>
# Paste the output here pd.show_versions() here
</details>
