Setting up DDE universal data Plugin with Python script as Server

zoringer · May 19, 2023, 12:57pm

Hello Everyone!
I continue creating data stream solution from Binance to Amibroker that was discussed here many times.
Streaming from a file generated by Python worked well except for "Permission denied error" on the Python side. I think is because Amibroker tries to read it same time Python write attempts.
So next upgrade solution would be to use DDE plugin.
After reading readme and thread1 and thread2 I run my python code:

import time
import win32ui, dde
from pywin.mfc import object as O_
import datetime
from datetime import datetime, timezone, timedelta


class DDETopic(O_.Object):
    def __init__(self, topicName):
        self.topic = dde.CreateTopic(topicName)
        O_.Object.__init__(self, self.topic)
        self.items = {}

    def setData(self, itemName, value):
        try:
            self.items[itemName].SetData(value)
            print(itemName+" "+value)
        except KeyError:
            print("error ")
            if itemName not in self.items:
                self.items[itemName] = dde.CreateStringItem(itemName)
                self.topic.AddItem( self.items[itemName] )
                self.items[itemName].SetData( str(value) )

serv_name='binance_'
t_name='BTCUSDT'

ddeServer = dde.CreateServer()
ddeServer.Create(serv_name)
#or serv_name+"|"+t_name+"!"+'Open' not working
ddeTopic1 = DDETopic(t_name+'!'+'Open')
ddeTopic2 = DDETopic(t_name+'!'+'High')
ddeTopic3 = DDETopic(t_name+'!'+'Low')
ddeTopic4 = DDETopic(t_name+'!'+'Last')
ddeTopic5 = DDETopic(t_name+'!'+'Time')
ddeTopic6 = DDETopic(t_name+'!'+'Volume')
ddeServer.AddTopic(ddeTopic1)
ddeServer.AddTopic(ddeTopic2)
ddeServer.AddTopic(ddeTopic3)
ddeServer.AddTopic(ddeTopic4)
ddeServer.AddTopic(ddeTopic5)
ddeServer.AddTopic(ddeTopic6)

while True:
    dat1='30000'
    dt_string = datetime.now().strftime("%H%M")#%H%M%S or %H:%M:%S  not working
    ddeTopic1.setData(t_name+'!'+'Open', dat1)
    ddeTopic2.setData(t_name+'!'+'High', dat1)
    ddeTopic3.setData(t_name+'!'+'Low', dat1)
    ddeTopic4.setData(t_name+'!'+'Last', dat1)
    ddeTopic5.setData(t_name+'!'+'Time', dt_string)
    ddeTopic6.setData(t_name+'!'+'Volume', '1')

    win32ui.PumpWaitingMessages(0, -1)
    print('running')
    time.sleep(3)

Above Python script output results:
DDE_2

and same time Amibroker not responding untill I close the Python script.

In the official docs every definition must consist of SERVER|TOPIC!ITEM - do I have to add SERVER| to the Topic?

Whatever I change it did not work.
Can anyone help plese?
Amibroker 6.18.0, DDE plugin version is 1.2.2.

zoringer · May 19, 2023, 1:32pm

DDE plugin version is 1.4.0

Tomasz · May 19, 2023, 3:06pm

No, it is because of your Python not being able to write.
What you should do is to write a file under NEW name (not used) and use shareDenyNone flag and after writing is complete you should RENAME file to final name. Renaming is atomic, while writing not.

Regarding DDE:

the LastPrice and/or Last Size and/or Volume must be changing
Time field must include ENTIRE timestamp YYYY-MM-DD HH:MM:SS

zoringer · May 19, 2023, 3:44pm

Hello Tomasz!
Thank you for shareDenyNone flag! Ill google how to set it to new file and then rename it!

Regarding DDE:
I changed the code (lower part):

vol=1
dt=30000
dat1=str(dt)
while True:
    
    if dt==30000:
        dt=30001
    else:
        dt=30000
    last=str(dt)
    vol=vol+1
    dt_string = datetime.now().strftime("%Y-%m-%d %H:%M:%S")#%H%M%S or %H:%M:%S  not working
    ddeTopic1.setData(t_name+'!'+'Open', str(30000))
    ddeTopic2.setData(t_name+'!'+'High', str(30010))
    ddeTopic3.setData(t_name+'!'+'Low', str(29999))
    ddeTopic4.setData(t_name+'!'+'Last', last)
    ddeTopic5.setData(t_name+'!'+'Time', dt_string)
    ddeTopic6.setData(t_name+'!'+'Volume', str(vol))

    win32ui.PumpWaitingMessages(0, -1)
    print('running')
    time.sleep(1)

So now it does as per your recommendations:
DDE_5

But still - Amibroker is not responding, but if I close Python - its back ok.

Cougar · May 19, 2023, 5:59pm

Adding to what Tomasz wrote,

With newer Windows, DDE is considered bad (malicious) like other things such as, Flash, etc. Honestly, DDE is obsolete, as it does not offer "sustainable" solution. It is available only in form of backward compatibility.

Since, you want to give Pythonic touch, my suggestion for you would be to use ODBC/SQL Universal Data/AFL plugin instead. Using Python you can feed a SQL Express Server, then from AmiBroker connect to that DB to enjoy both the worlds of Data and AFL plugin. That would not only be convenient but also open several effective, flexible doors for you.

Otherwise, if you are willing to invest some time to dive deep, then writing a DLL data-plugin is not that difficult either. Yes, one would need basic knowledge of C++, C# or Rust. The process is explained in ADK (with C++ examples), it is as simple as fitting two ends of a pipe! Assign respective ADK structures with your corresponding datafeed API (mostly WSS), abide by the rules as explained in it, that's it!

zoringer · May 19, 2023, 9:10pm

Thanks a lot for ODBC/SQL! I come to same ideas to investigate it. Regarding ADK - I also consider to dive deep there but later. I remember Tomasz recomendations - to use Ami standard options before goint into ADK C++ works.

Cougar · May 19, 2023, 10:43pm

You're welcome!

It's pretty straight forward!

Binance API is pretty juicy + good doc.

And you've pyodbc for C, R, U (most of the times) and D (bad ticks if any), rest AB's configured ODBC plugin gives O₂ to your breathing charts, eh!

zoringer · May 20, 2023, 11:05am

Thanks a lot Cougar!
(A bit offtopic) Can you advise as well how would you do Amibroker-binance trading solution?I read some few threads here and not sure if anyone succeded to send any order to binance directly from ami. Additionally my trading system also needs to send up to 50-70 orders at once(trading at 1m tframe). I can do it from python script for example. My idea was to use odbc or write/read by files. Can you recommend your vision on that?

rottor · May 20, 2023, 12:56pm

to send orders directly from AFL you'd need to compute a cryptographic hash and signature of your request

I've done it in the past from AFL but with external helper utility

also, if you are not aware - there are some plugins for crypto/Binance out there (some of them free), have you tried them already ?

Cougar · May 20, 2023, 3:19pm

Binance provides an official Python connector. Before anything else, you must be thorough with given examples in that repo.

ODBC is for data (consider these examples for feeding SQL Express), that's got nothing to do with order placements (consider these examples to trigger orders).

There are many ways!

To keep things pythonic, simplest & effective implementation can be done by creating an Asynchronous server using FastAPI segregating all your intended tasks (such as place new, cancel, modify orders) onto respective routes. The server is set to run on uvicorn, like so:

uvicorn.run( 'myBinanceApp:app', host = '127.0.0.1', port = 5000 )

Here, the server runs on localhost:5000, so, you could now basically hit http://localhost:5000/<specify your task route> with respective URL-params (specifying Symbol, Quantity, Price, OrderType, etc). Each of those routes are assigned to capture URL-params for its corresponding Request (GET, POST...) to Binance (or any other) API Endpoints.
For example, a new Order route's structure would somewhat look like:

from binance.websocket.spot.websocket_api import SpotWebsocketAPIClient

from fastapi import FastAPI, Request
from fastapi.responses import RedirectResponse, HTMLResponse
from starlette.middleware.sessions import SessionMiddleware

app = FastAPI()
app.add_middleware( SessionMiddleware, secret_key = 'mysecret' )

@app.get( '/' )
async def index( request: Request ):
    return HTMLResponse( 'Hello there! server is up & running...' )

@app.get( '/newOrder' )
async def order( request: Request ):
    symbol   = request.query_params.get( 'symbol' )
    side     = request.query_params.get( 'side' )
    type     = request.query_params.get( 'type' )
    quantity = request.query_params.get( 'quantity' )
    price    = request.query_params.get( 'price' )

    my_client = SpotWebsocketAPIClient(
        stream_url  = "wss://testnet.binance.vision/ws-api/v3",
        api_key     = api_key,
        api_secret  = api_secret,
        on_message  = message_handler,
        on_close    = on_close,
    )

    my_client.new_order(
        id               = id,
        symbol           = symbol,
        side             = side,
        type             = type,
        timeInForce      = "GTC",
        quantity         = quantity,
        price            = price,
        newClientOrderId = generateSeqOrderNum(),
        newOrderRespType = "RESULT",
    )

Once your server is up & running, upon satisfying trade logics (discretionally or systematically automated) AmiBroker places new order(s) through InternetPostRequest( url ), where:

url = StrFormat( "http://localhost:5050/newOrder?%s=%s&%s=%s&%s=%s&%s=%s&%s=%s",
	"symbol",symbol,
    "side",side,
    "type",type,
    "quantity",quantity,
    "price",price
);

After few months once you become fluent, using NGNIX you may host that Python-based backend server (maybe along with a ReactJS based frontend interface) to AWS or anything ( Linode) for 24/7 anytime, anywhere access. Then, you may place order to https://zoringer.com/newOrder from AmiBroker relaxing on a Greece beach.

peace-laptop

P.S. Before jumping onto coding, writing algorithms (in pen-n-paper) is essential. It is also necessary to test all API (functionalities + routes) in Postman beforehand. That would substantially help you plot a flawless map!