Don't mess with watch list files directly, was: Watchlists disappearing

[SpandexMan]

I use a python script to get ticker lists that I have permissions to trade.

from binance.spot import Spot
import config
import pandas as pd
import logging
import eventlog
import time

logging.basicConfig(level=logging.DEBUG, filename='log.txt', filemode = 'a')
eventlog.timelogger()
eventlog.scriptid('ExInfo.py')

Client = Spot(config.apiKey, config.apiSecret)
exInfo = Client.exchange_info()
sym=exInfo['symbols']
df = pd.DataFrame(sym)
trad = df[df['status'].str.endswith('TRADING')]
trad.to_csv('pairs.csv')
df2 =pd.read_csv('pairs.csv')
allowed = df2[df2['isSpotTradingAllowed']==True]
allowed.to_csv('pairs.csv')
df3=pd.read_csv('pairs.csv')

symList = df3['symbol']

symList.to_csv('C:\Program Files\AmiBroker\Databases\Binance\WatchLists\symbolList.tls', index=False, mode='w')
time.sleep(1)
df4 = pd.read_csv('C:\Program Files\AmiBroker\Databases\Binance\WatchLists\symbolList.tls')

usdt= df4[df4['symbol'].str.endswith('USDT')]
usdt.to_csv('C:\Program Files\AmiBroker\Databases\Binance\WatchLists\PairsUSDT.tls', index=False)
btc= df4[df4['symbol'].str.endswith('BTC')]
btc.to_csv('C:\Program Files\AmiBroker\Databases\Binance\WatchLists\PairsBTC.tls', index=False)
eth= df4[df4['symbol'].str.endswith('ETH')]
eth.to_csv('C:\Program Files\AmiBroker\Databases\Binance\WatchLists\PairsETH.tls', index=False)

If I open Amibroker, the watchlists are there, ...

but the moment I either save database, or even close amibroker, the watchlists are deleted...

How do I fix this? Thanks

[Tomasz]

First, direct manipulation of database files is NOT SUPPORTED.

There are functions INSIDE AmiBroker that you should use
https://www.amibroker.com/guide/h_watchlists.html

AmiBroker manages watchlists by itself and you should not mess with that. At any time in the future, the layout, content and inner workings of database files might change without warning and your code WILL BREAK.

Having said that, your python code is incorrect, because it does not update index.txt file which is CRUCIAL as it contains the watch list names in correct order (necessary to know which watch list is first, second and so on), and .TLS files alone are just NOT enough.

Also symbols must exist in the database. It is not enough to place symbol in TLS. It must be present in the database already or added (Symbol->NEW)

[SpandexMan]

I have tried now using batch and creating custom ASCII importer formats using the wizard (
When I created the uploader formats I specified 'no quotes', skipped my first line and had only 1 column, i.e ticker. These are essentially empty, waiting to later receive OHLCV data as Import a new file periodically. ), I then use the batch for importing them. When I run the batch, the watchlists populate.
Before;

After;

However when I close Amibroker then re-open, the watch lists have disappeared. They disappear when I either close amibroker, or I manually save the database (i.e they are deleted from the Watchlists folder in the Binance Database.

Same thing happens when I run extrenally...

I get no database next time I open.
The database is empty (apart from a single symbol BTCUSDT). The plan is, once I have my watchlists, I will periodiclaly make a call to Binance to update the various lists (limit is 1200 per minute, so I'd make a couple of clals over 2 minutes depending on the length of my watchlis, then save the file as a CSV and import that to the database)

I'm happy to delete the database and startover if I have corrupted it in some way but before I do all that, I want to know that I am on the right track.

What am I doing wrong?

[SpandexMan]

Wow, thanks for the quick reply. Thanks Tomasz.
Ok, noted on the index file. I'll make some ammendments. I'm looking to dynamically update my watchlists on a periodic basis so that I am only capturing symbols where I actually have permissions.

Is there was way I can access the importing of watchlists from shell?

I tried to workaround by using my python script to instead save the watchlists in the amibroker folder amd import from there, then I tried in shell;

.\broker.exe /database "C:\program files\amibroker\databases\binance" /add "C:\Program Files\AmiBroker\AmiQuote\PairsUSDT.tls" /SaveDatabase /exit

but that doesn't give me what I am looking for. Is there an argument I am missing?

[Tomasz]

As I wrote: index.txt is important, it must contain the list of all .TLS files inside Watch lists folder in correct order (index.txt is used by AmIBroker to know the order of watch lists). And symbols added artificially to .TLS must exist in the database itself.

FWIW: I don't know from where you got that "/add" switch. It does not exist in AB.

[SpandexMan]

Thanks for that. I’ll play around with the index file. I can manipulate that easy enough.

Re the ‘add’ switch, I misinterpreted the OLE documentation. I’m new to OLE, and amibroker, and programming in general. Import was not working. (Ami broker was opening, but that was all)

I’ve got no idea how to control Amibroker from outside using shell and batch and I mostly just try and throw together some tricks that I have found on this forum. I had thought I’d found it in the AmiBroker Object Model page but if I to be entirely honest, the documentation is way beyond my level of comprehension.

Thanks again, I can work with that.

[Sean]

Before you continue, you should get your money off the exchange, as it might not be solvent in a week.

[SpandexMan]

Hi Sean

I had noticed, thanks for the idea. I don’t have funds in there at the moment apart for some ‘testing’ funds which I can afford to lose and I agree, that’s certainly possible.

A lot of the libraries in cryptoland are fairly similar to the python-Binance library, so a lot of the stuff I’ve learned and work I’ve done to integrate my Amibroker set up into Binance over the last 4 months hopefully won’t be completed for nought.