WsRtd - Websocket-Json standard Data plugin - Test Release

Plug-ins
63

Kindly do not modify the relay.py, it will break your flow in the future as it is a core component.
That is why there is client.py in which you write the code. All vendor_library comm is placed here.

When BrokerWS, ArcticDB etc gets integrated, relay logic is only going to increase in complexity.

in future, it may be made into a service once stable, and some UI shortcuts for it. I know right now you need 1 terminal to keep it running. But it is good as you can see some debug messages instead of running it in the background.

2 Likes
64

One more query.

Are these the list of commands or queries we get from WsRtd plugin?

  1. bfall (BackFill)
  2. bffull (BackFill)
  3. bfauto (BackFill)
  4. addsym (AB/user requesting new symbol)
  5. remsym (AB/User request to delete symbol)
  6. bfsym (Backfill)

Is there any other communications our dll plugin can do? I know you are working on a full scale system but just wanted to see if this in pipeline or would be a bad idea to add new some more additional commands which can be used for other purposes.

65

@nsm51 It is great to see that we have got customised plugin from you. It is very great initiative. I am also a non programmer trying to understand AFL and getting realtime data from the broker. I was forcebly opened my IB account for the sake of data but now i think i can integrate from my broker. Can you please share me the dll please.

1 Like
66

If you have something in mind, I am adding it to the feature list if it is compliant.

I'm working on BrokerWS right now and it is a pain. not c++, the json standard because every broker has a different implementation.

1 Like
67

Why is it such a big deal with the broker interface?
Isn't BrokerWS supposed to be a standard from AB to json and back?

I mean, why do you have to worry about the brokers?

68

There are many fields, and AB itself supports Bracket orders for example. so it should be wholesome and I don't have old code like data plugin to test.

In client-app you don't need to maintain so much state like you have to in TI, only the recent bar is enough in one dataframe.
In the client-app order_class_wrapper i am writing, there are 4 DFs.

Anyway, code is submitted to AB for review/approval.

2 Likes
69

So you mean you have to maintain an array of orders?

Btw, an order can be filled or not, or even partially filled. Can this be accounted for? Or AB presumes if it sends an order it will be filled? Is there an async mecanism to update the order's state?

70

IBController 1.3.8 READ ME

exact implementation here, it is a separate windows appl
run BrokerIB.exe shipped in your installation

2 Likes
72

Can you please share access to the .dll file (I could not find a way to DM you)? Thanks.

73

DM'd the instructions for testing

2 Likes
74

Thank you. I'll download it and test it out.

75

Hi @nsm51 do you develop custom plugins? I have a project I'd like to discuss

76

@nsm51 Wanted to drop in and say thank you for the development of this plugin. I've successfully configured minute bars with quote tick updates to price using BarCharts API. Awesome work! I do have a quick question...

Here's a json message amibroker would receive from my relay:

[{"n": "$IUXX", "t": 85500, "d": 20250226, "o": 21344.51, "h": 21350.52, "l": 21335.92, "c": 21339.02, "v": 0, "do": 21170.51, "dh": 21348.01, "dl": 21140.55}]

In the RTQ Window, the percent change field doesn't update. I am not super familiar with the RTQ window, so maybe this could be solved by sending into one of the Aux? What might I be doing wrong?

1 Like
77

The spaces, they are not there (usually) in default json encoding. Turn them off.
this json match will fail (It's in the manual for performance reasons)

Also, for RTD try and send an array [{"n":"sym1",},{"n":"sym2",},{"n":"sym3",}...]

2 Likes
78

Thanks for the tips! This is my updated format for json messages to Ami:

[{"n":"$SPX","t":105700,"d":20250306,"o":5732.14,"h":5733.7,"l":5730.41,"c":5730.41,"v":0,"do":5785.87,"dh":5812.08,"dl":5715.33,"bp":5728.43,"ap":5732.3,"bs":0,"as":0},{"n":"$IUXX","t":105700,"d":20250306,"o":20081.55,"h":20092.42,"l":20077.55,"c":20077.55,"v":0,"do":20231.74,"dh":20473.4,"dl":20012.69}]
[{"n":"$SPX","t":105700,"d":20250306,"o":5732.14,"h":5733.7,"l":5730.28,"c":5730.28,"v":0,"do":5785.87,"dh":5812.08,"dl":5715.33,"bp":5728.37,"ap":5732.27,"bs":0,"as":0},{"n":"$IUXX","t":105700,"d":20250306,"o":20081.55,"h":20092.42,"l":20077.49,"c":20077.49,"v":0,"do":20231.74,"dh":20473.4,"dl":20012.69}]
[{"n":"$SPX","t":105700,"d":20250306,"o":5732.14,"h":5733.7,"l":5730.26,"c":5730.26,"v":0,"do":5785.87,"dh":5812.08,"dl":5715.33,"bp":5728.25,"ap":5732.11,"bs":0,"as":0},{"n":"$IUXX","t":105700,"d":20250306,"o":20081.55,"h":20092.42,"l":20076.98,"c":20076.98,"v":0,"do":20231.74,"dh":20473.4,"dl":20012.69}]

The % Change field still doesn't update in the RTQ Window... any other potential fixes?

79

% change is calculated from previous day closing, which AB does, not the plugin.

So you need to supply "pc" key. Prev close.

The way RTD works, every packet must contain dh,do,dl,pc etc even though it doesn't change.

1 Like
80

thank a lot for the development of this plugin.
I tested client and server on terminal working well.I played with broker api like zerodha, upstox
Can you guide me Wsrtd.dll Where I can find ?
Want to contribute connect broker with amibroker.

81

I've sent DM with test instructions

1 Like
82

Can I just say 1000 thank yous @nsm51 this is a phenominal piece of work.

I'm populating charts and calling the RTD on 1 minute data and that is working well.
What I am trying to achieve is to reduce the granularity the further back in time I go. So I'd call 1000 X 1min candle, then 1000 X 5 min candles prior to that, and so on.

When I broadcast these sequentially, often only 1 of them randomly will save, but usually the last one. I'm calling the following in a loop (updating the start timestamp by minusing the number of datapoints * length in ms of each datapoint) The data is correct, but I'm not getting it into Amibroker.

data = await exchange.fetch_ohlcv(
                                                symbol=symbol,
                                                timeframe = timeframe,
                                                since = timestamp
                                                limit = 1000
                                                )
            formatted_data = [
                [
                    int((datetime.datetime.utcfromtimestamp(row[0] // 1000)
                         .replace(tzinfo=pytz.utc)  
                         .astimezone(self.timezone)
                         .strftime("%Y%m%d"))),  # Date as integer (YYYYMMDD)

                    int(datetime.datetime.utcfromtimestamp(row[0] // 1000)
                        .replace(tzinfo=pytz.utc)  
                        .astimezone(self.timezone) 
                        .strftime("%H%M%S")),  # Time as integer (HHMMSS)

                    *row[1:],  # OHLCV values
                    0  # Extra 0 at the end
                ]
                for row in data
            ]
            json_dict = {"hist":f"{symbol_and_exchange}","format":"dtohlcvi"}
            json_dict['bars'] = formatted_data
            json_string = json.dumps(json_dict)
            self.broadcast(json_string)

Is this more likely a blocking thing (I am not using Queue,... but I intend to have a look at that in coming days) , or should I not send multiple jsons sequentially with the same values (apart from bars).

Any guidance, as always, much appreciated.

1 Like
83

AB does not recommend this, even though it works. (abuse it correctly)
Data should be supplied only in the base time interval.

Now what is happening is, the plugin DB stores only one json-hist backfill packet. This is why only the most recent one is kept. IF data is in plugin-db AND a NEW json-hist pkt arrives for the same symbol, it overwrites the previous one.

NOW, if you want to prevent this, you need AB to call that symbol. Data from plugin-DB goes to AB DB "ONLY" when AB choses.
so try this, send json-hist packets per symbol,
run exploration for all those symbols,
then send more json-hist,
then run exploration.
This is how AB <-> data plugin interface is designed. Ensure sufficient bars in DB settings.

2 Likes