The data that we're dealing with is like price data of stock prices, and here's the situation.
How can I implement this data that is poured out in order without duplication?
'''MAX_TABLE_LENTH = 200'''
def recent_trades():
return data['trade']
First of all, there is a function preconfigured in this way
At first,
while(true):
re_td = recent_trades()
csvWriter.writerow(re_td.values())
As shown in the same way, I coded the list in one variable and recorded it in a csv file.
1~12, 1~34, 1~60... (omitted) 1 to 169, 1 to 200
He kept saving all transactions from the beginning until 200 transactions were piled up
Since then, 200 have accumulated, so if you erase the last 100 again
101~131, ... Keep redundant data, such as 101~186, 101~200, from start to finish.
If you don't have a new contract, keep saving the same contract details indefinitely.
In less than three minutes, I got more than 200 MB of duplicate data.
How should I write the code so that I can save only the newly added tightening details, excluding the tightening details that I have already saved, without having to search again from 1?
I ask for advice on the data structure or program structure that I have to use.
python websocket list
Is there a duplicate check logic ? Check the data['trade'] for an identifier such as an eigenvalue or timestamp increasing by increment. If you have it, you should use it.
For example, if the downloaded data looks like this:
[
{
id: 399865, // Assume this value is unique and always large
stockId: 'APPL',
dealType: 1,
stockUnit: 80,
stockUnitVal: 500,
updatedAt: '2018-07-30 11:50'
},
...
]
This is what the pseudo code for double checking will look like.
lastId = 0 // id value of the last processed data
for deal in data['trade']:
If (deal.id > lastId): // If you get a larger value than before,
DoSomething() // and do the work
lastId = deal.id // Save Now as Last Processed
© 2024 OneMinuteCode. All rights reserved.