Starting Data plug in project

I’m going to get started in developing a simple data plug-in for myself.

I’m nearing the end of a course in C++ and I’ve read the ADK and understood/retained about 80-90% of it.

The instructor in the course teaches, (amongst other things)pointers, for the purpose of being able to work with code from older projects, but says that passing function by reference (e.g &i) is nowadays used to achieve the same job, and the programmer does not have to be as concerned with allocating and then releasing data on the ‘heap’, and memory leaks.

I notice that in the ADK, the examples mainly use pointers.

Am I able to write my functions passing parameters by reference instead of using pointers and de-referecing,
For the purpose of this project?

Secondly, I don’t understand the manual precisely. I know how to call and listen to Websockets in Python, and I know how to create files from them. I don’t know how valuable it would be to redo that work in C++, if I am able to continuously wrote and overwrote files using my Python scripts (running concurrently)

If I am going to use a file based approach, does the data sit in the file (e.g BTCUSDT.AQI) and continue to fill up (appended) until it is called (e.g by a Notify() call being sent from AmiBroker), an array is sized and memory allocated, data read and then imported into AmiBroker (and can then be safely deleted while a new file is filled up) essentially meaning that the files are a kind of, perhaps a kind of ‘buffer’? Or, is set up so that single line files are created, (is there a way of or is it even desirable calling Notify() here?) and then the AQI file is imported and deleted?

The reason I ask, is that I want to get the file creation side of the project right before I get started. I’d love to hear thoughts.

The third, and perhaps this is a stupid question( in fact I rather suspect it is) I notice that character arrays are used in the sample (e.g char[256]…). There’s no reason that I haven’t noticed, that I couldn’t simply use strings, is there?
Thanks for your wisdoms.

Spandy.

see if this helps for your 3rd Q
What are the Differences Between C++ Char*, std:string, and Char[]? - Scaler Topics

1 Like

No, you can't use reference when function expects pointer. References and pointers are different things (despite the fact that under the hood they are just addresses).
And the whole "oh whoa no, scary pointer" nonsense spread by "instructors" is total BS. Pointers are first class citizens in any C/C++ program, also in 2024. Any dynamic memory allocation requires pointers. There are things that are doable only using pointers.

std:string is allocated from heap it way slower than using char array local variable (that lives on stack and such 'allocation' takes literally zero time), also entire std was truly broken in older versions of VC++ (6.0) and avoiding it was good idea back then when this example was written.

FYI: in underlying code std:string uses "whoa oh no" pointers internally. Every object that has structures allocated from heap IS USING POINTERS under the hood.

Also in plain C you only have pointers, not references. Plain C API and raw pointers make it possible to interface to other languages. References are not portable across different languages.

1 Like

I can learn to work with that. Thanks for the useful pointers. :face_with_hand_over_mouth:

I'm using visual studio 2022. Ive tried loads of configurations and am just digging myself further down the rabbit hole. I'm really getting stuck.

I have opened the repository and Cmake generated what it does.

I've installed visual studio 3 times and removed replaced the ADK library twice, worked through loads of errors and configurations, stack exchange posts and I'm consistently getting stuck.

I am using the 'Data_Template repository in the ADK and have created a project in visual studio 2022 for a dll. It doesn't create MyPlugin.dll, StdAfx .h and StdAfx.cpp, but instead created dllmain.cpp, pch.h and pch.cpp.

I have tried variations of including StdAfx.h, not including StdAfx.h, I have tried changing the contents of pch.h so that they are actually what was in StdAfh.h (and then not including StdAfx.h)

I am currently returning errors;

Severity Code Description Project File Line Suppression State Details
Error LNK2005 DllMain already defined in dllmain.obj Spandex C:\Users....\source\repos\AmiBroker ADK Sandbox\Samples\Data_Template\Spandex\mfcs140ud.lib(dllmodul.obj) 1

and

Severity Code Description Project File Line Suppression State Details
Error LNK1169 one or more multiply defined symbols found Spandex C:\Users....\source\repos\AmiBroker ADK Sandbox\Samples\Data_Template\Spandex\ARM64\Debug\Spandex.dll 1

I don't know what to do next. I can generate the data files in the exact configuration as suggested in the ADK, and again, oldest quote at top, newest bottom. It's this compilation of a plugin to read those data files that I am havig immense difficult with.

When you are using ADK, you don't need to (and should NOT) create a NEW project. You should open existing one instead. What you did by "creating a new project" was to create a NEW files (Visual Studio did that for you) that contain COPIES (unnecessary definitions) of already existing DllMain() function.

1 Like

Ok, so now my understanding then is that I will compile from the project created by the cmake? Is that correct?

Yes CMAKE prepared projects for you. You just need to open already existing projects.

1 Like

That is superb!! Thanks so much. It compiled and I am getting data in now.

I have a new question and I am uncertain if it is related to this or belong in another thread, but I'll ask it here.

I haven's actually modified the source code except to change the directory so that it is not relative. My GetQuotesEx function is basically what is in the sample.
Seperately, I have a python script listening to a websocket and writing to the ASCII folder. (Anyone feel free to use it;...)

import websocket
import json
import pandas as pd
from datetime import datetime
import streamUSDT1

def on_open(_wsa):
    print("On open function")
    data = dict(
        method='SUBSCRIBE',
        id=1,
        params=streamUSDT1.dictionary
    )
    _wsa.send(json.dumps(data))

def on_message(_wsa, data):
    df = pd.read_json(data, orient='index')
    data2 = (df[0]['k'])
    df2=pd.DataFrame(data2, index = [0])
    ticker = (df2['s'][0])
    candleOpen = (df2['t'][0])
    normTimeCandleOpen = str(datetime.fromtimestamp(candleOpen/1000))
    indSpace=normTimeCandleOpen.find(' ')
    dateVal=(normTimeCandleOpen[2:indSpace])
    dateVal=dateVal.replace("-", "")
    timeVal=(normTimeCandleOpen[indSpace+1:-3])
    timeVal = timeVal.replace(":","")
    openPrice = (df2['o'][0])
    highPrice =(df2['h'][0])
    lowPrice =(df2['l'][0])
    closePrice =(df2['c'][0])
    volume =(df2['v'][0])
    titled_columns={'Date':dateVal,
                    'Time':timeVal,
                    'Open':openPrice,
                    'High':highPrice,
                    'Low':lowPrice,
                    'Close':closePrice,
                    'Volume':volume}
    df2=pd.DataFrame(titled_columns, index=[0],columns=None)
    df2.to_csv( "C:\\Program Files\\AmiBroker\\ASCII\\"+ticker.upper()+".AQI" ,header=False, index=False, mode='a')
    df3=pd.read_csv("C:\\Program Files\\AmiBroker\\ASCII\\"+ticker.upper()+".AQI",header=None,index_col=None)
    try:
        prior_last_row = df3.loc[len(df3)-2]
        existingDateVal = (int(prior_last_row[0]))
        existingTimeVal = (int(prior_last_row[1]))
        if (existingDateVal == int(dateVal)) and (existingTimeVal == int(timeVal)):
            df4 = df3.drop([len(df3)-2])
            df4.to_csv("C:\\Program Files\\AmiBroker\\ASCII\\"+ticker.upper()+".AQI" ,header=False, index=False, mode='w')
        else:
            pass
    except:
        pass

def run():
    print("Running Websockets")
    stream_name = 'some_name'  # this is not important
    wss = 'wss://stream.binance.com:9443/ws/%s' % stream_name
    wsa = websocket.WebSocketApp(wss, on_message=on_message, on_open=on_open)
    wsa.run_forever()

if __name__ == '__main__':
    run()

I am getting Data into the charts :grin: . I can manually refresh the charts with pressing F5 on the keyboard, but if I either set

RequestTimedRefresh(5, False);

or if I run Explore on auto repeat, the arrays are not being refreshed.

How should I address this? Do I need to address this on the plugin side for example calling 'WM_USER_STREAMING_UPDATE' in a while loop (say every 2 seconds) or is there something else I should pay attention to?

Thanks in advance.

Look at the examples, it is all there. There is a QuoteTracker plugin that does rt updates.

Yes, you need to send WM_USER_STREAMING_UPDATE if you want to tell AMiBroker that you have new data. This has to be done properly, as described in many posts on this forum

https://forum.amibroker.com/search?q=WM_USER_STREAMING_UPDATE

Thanks for the direction. I have read those forum posts.

Please forgive my ignorance. In the ADK readme, the distinction is made between local 'file based' and remote sources.

I have reviewed the QT example and on line 622, inside a CALLBACK OnTimerProc() function, is written

::SendMessage( g_hAmiBrokerWnd, WM_USER_STREAMING_UPDATE, 0, 0 );

I have read about SendMessage. I'm using the Data_Template sample along with ASCII files, so I don't have such a CALLBACK function (like the QT example). I'm not clear how to send WM_USER_STREAMING_UPDATE to Amibroker.
My attempts so far;
I have declared and defined a new function, WMUserLoop() as

PLUGINAPI void WMUserLoop()
{
	while (true) {
		::SendMessage(g_hAmiBrokerWnd, WM_USER_STREAMING_UPDATE, 0, 0);
		Sleep(2000);
	}
	
}

and I have called this function when the plugin is initialised, like this;

PLUGINAPI int Init(void) 
{ 
	AFX_MANAGE_STATE( AfxGetStaticModuleState() );
	WMUserLoop();
	return 1;
}

having earlier declared

HWND g_hAmiBrokerWnd = NULL;

This causes the plugin to freeze/ not load. I understand why but my rationale was that I'd call the function upon initialization and then keep it in a loop.

So I have tried instead to place

::SendMessage(g_hAmiBrokerWnd, WM_USER_STREAMING_UPDATE, 0, 0);

inside Notify() like this;

PLUGINAPI int Notify(struct PluginNotification *pn) 
{ 
	AFX_MANAGE_STATE( AfxGetStaticModuleState() );
	::SendMessage(g_hAmiBrokerWnd, WM_USER_STREAMING_UPDATE, 0, 0);
	return 1;
}	 

(again having earlier declared HWND g_hAmiBrokerWnd = NULL;)
This doesn't work as Notify().

I've also tried including a new Notify script inside the .dll, called Notifier.cpp

#include "stdafx.h"
#include "Test.h"
#include "Plugin.h"
#include "resource.h"
#include "ConfigDlg.h"


HWND g_hAmiBrokerWnd = NULL;
int main() {
	while (true) {
		::SendMessage(g_hAmiBrokerWnd, WM_USER_STREAMING_UPDATE, 0, 0);
		Sleep(2000);
	}
	return 0;
}
int main();

Am I calling SendMessage correctly, should I be calling it as part of another function? I think, if I am correct, that my project is a little different to the QT example in the ADK.

I'm a little lost. I was thinking of just copy pasting the GetQuotesEx() function from my Data_Template of ASCII and replacing the one in QT but there are a lot of other functions in QT that I am assuming have inter dependency one another.

My solution is not exactly a realtime streaming data plug in. I am capturing the OHLCV on the 1 minute time frame (like the ASCII example). My files are stored locally but 're-created' every 2000 ms on API call (managed by a separate python script). I have not been streaming Bid, Ask etc so have no need for the RecentInfo() function. Can I still send WM_USER_STREAMING_UPDATE if I am using a file based solution and if so, what is the best way to do it?

Can I simply implement the RecentInfo() funciton, leaving it without any assigned parameters, but place my ::SendMessage( g_hAmiBrokerWnd, WM_USER_STREAMING_UPDATE, 0, 0 ); If so, how do I pass ticker to WPARAM and recentInfoStructure to LPARAM?

Sorry, but I cannot provide free support with such amount of "spoon feeding".
AmiBroker free support does NOT cover C++ programming or Windows programming nor using Visual Studio, etc.
You need to study the docs (ADK) and the forum and figure out things yourself using plenty of information available on the web on subjects like using Visual Studio, writing DLLs,

Few pointers:

  1. WM_USER_STREAMING_UPDATE
    Please read the other messages I wrote about WM_USER_STREAMING_UPDATE on this forum. I explained everything a number of times already. In principle you should properly fill all params if you want to get real-time updates. If you don't send necessary information, your updates will be delayed. In short the message with WPARAM and LPARAM set to zero is "lazy" update (charts will update at slower pace and not necessarily at the time of the call, but later). If your source is delivering data only every 2 seconds, it would be enough.

    It should NOT be called NOT from Notify() function.

    Notify is an export that is called on certain situations like right clicking the status bar plugin area. JUST READ THE DOCS, it is all written there.

    WM_USER_STREAMING_UPDATE should be either called from timer (SetTimer) or better from callback that your data source calls when it receives new data. Again: look at the supplied examples to find out when / where to call SendMessage.

  2. Your main() function is wrong on so many levels:

  • DLLs do not have main()
  • main() is only for command line programs and they should NOT be calling SendMessage at all
  • don't do Sleep() in Windows apps. Windows needs to process messages, apps should not sleep. They should be handing message queue. Windows apps should use timers if they need to wait.

Sorry to say, but I need to reiterate that writing DLLs is not really for beginners: 5 reasons why you should NOT write DLLs

2 Likes