AmiPy plug-in - Python integration

AmiPy plug-in: AmiBroker-Python integration

UPDATE September 24, 2022: Version 1.0.1 released!

With permission from @Tomasz I am announcing here the next version of my AmiBroker-Python integration plugin. This plugin allows you to call Python functions from AFL side. This opens out possibility to use wide selection of Python libraries such as numpy, scilearn, tensorflow and such in AmiBroker.

It takes a great amount of time developing a program for public use so if You are using this plug-in on every-day basis please consider making a donation. The link below allows you to send small amount of money ($5/$10/$25 of your choice) to me. Thank you very much for your generous support.

Click here to donate by PayPal

PS:
I don't have any more ideas what to add to this plugin so it may be that this will be the final version of it.
If You have any idea what could be enhansed or added feel free to post it on this thread.

If you want help with anything use the lastest version of AmiPy

Download link:

AmiPy 1.0.1: https://ev1.pl/amipy/AmiPy_1_0_1.zip (64 bit only)

VirusTotal file check 100% clean

AmiPy Changelog: https://ev1.pl/amipy/ChangeLog.html

Minimum requirements

AmiBroker version 6.30 or higher x64 (64-bit only)

Installation

  1. Install Python 3.x x64 version where x>=7 (3.7 or greater) (newer are encouraged) (Download Python | Python.org)

  2. install numpy module: pip install numpy (in cmd)

  3. from AmiPy.zip extract AmiPy.dll into "Plugins" subfolder of AmiBroker 64-bit

AFL functions

PyLoadFromFile( context, fileName )

Argument Type Description
context string Name of the context
filename string Path to file that should be loaded. Can be raw code (.py file) or compiled code (.pyc file).

Return value: Does not return any value

PyEvalFunction( context, PyfunctionName, ... [fn args] )

Argument Type Description
context string Name of the context
PyfunctionName string function name (name of callable object) definied in global scope.
... [fn args] float / string / array arguments used in call of python function1

Return value: return value of called function2

1AFL to Python variable conversions:

AFL type Python type
number float
string string
array 1-D numpy array (np.ndarray)

2Python to AFL variable conversions:

Python type AFL type
number convertable to float (e.g. int,float,bool) number
string string
1-D numpy array (np.ndarray) of size of AFL array and convertable to float array array
2-D numpy array (np.ndarray) convertable to float array matrix

You can also return numpy scalar containing any mentioned above type.


Python module - AmiPy

This Python build-in module allows to call specific AmiBroker functions from your python program.

WARNING: This is a build-in module which means that it is only accessible, when running python from the plugin. That means you will not be able to use and/or import this module when started from command line/VS Code etc. It also means that you cannot install it (e.g. by pip) to external python implementation.

AmiPy methods

Print( str )

prints string in AFL Debugger

AFL equivalent of printf( "%s", str );

Error( str )

prompts AFL error with content string

AFL equivalent of Error( str );


Initialization file

After first time AmiPy is used in AmiBroker's working directory should be a initialization file: AmiPy.ini


[Logging]

; where log file is stored, default: AmiPy.log

Path=<string>

; If given event is logged

; default: error, warning and main_ev are true and all else false

; when you want support you need to set all below mentioned options to true but for `debug` and for crush reports debug must also be set to true

error=<bool>

warning=<bool>

fn_start=<bool>

fn_end=<bool>

main_ev=<bool>

debug=<bool>


Common problems

  1. Problem:
  • PyLoadFromFile prints error: "Cannot start array API (no numpy?)".

  • Giving PyEvalFunction array as argument or returning from called function array-like object prompts error "Exception caught in external DLL [...]".

    Solution:

  • Install numpy on your device on the version of python denoted in AmiPy.log.

  1. Problem:
  • PyLoadFromFile and PyEvalFunction not showing in AFL

  • In AmiBroker's Tools->Plug-ins window AmiPy doesn't show up.

    Solution:

  • Ensure that you have python 3.x (where x>=7) installed

  • Ensure that you have python (python3.dll) in PATH

  1. Problem:
  • AmiPy does not load correct version of python

    Solution:

  • Ensure that AmiPy sees python3.dll of the version that you want to load before any others. The simplest way to do this is to copy it from python's to AmiBroker's directory.

Tips

Sometimes Python modules take significant amount of time to load and/or initialize. In such situation it may be worthwhile to load/initialize only once.

To load part of file only once (e.g. to create variables) you can use in Python code:


import numpy as np

import AmiPy

if '__' + __file__ + '_initialized' not in globals():

  globals()['__' + __file__ + '_initialized'] = True

 

  # ... put your time-consuming initialization code ...

  # ... so it is run only once here

  #  

# ... your code ...

Examples

  1. Evaluating simple functions

AFL:


PyLoadFromFile("example1", "python.py");

Close2 = PyEvalFunction("example1", "funct", Close);

Diff = PyEvalFunction("example1", "diff", High, Low );

Plot( Close, "Close" );

Plot( Close2, "Close + 2" );

python python.py:


import numpy as np

import AmiPy

def funct( array ):

    AmiPy.Print( "This message was sent from Python!\n" )

    return array + 2

   

def diff( a, b ):

    return a - b

39 Likes

For clarification pip install numpy means to enter such command to Windows command line (cmd). It installs Python module numpy needed for array and matrix processing.

Can you add some example of using?

AFL:

PyLoadFromFile("python.py");

Close2 = PyEvalFunction("Add2", Close);
fibbonaci = PyEvalFunction("Fibb", 2);

Plot( Close, "Close" );
Plot( Close2, "Close + 2" );

Python python.py:

import numpy as np
import AmiPy

def Add2( array ):
    AmiPy.Print( "This message was sent from Python!\n" )
    return array + 2
def Fibb( n ):
    mat = np.array( [[1, 1],[1,0]] )
    fibbMat = np.linalg.matrix_power( mat, int(n) )
    return fibbMat[1][0]
4 Likes

Thank You for your work, but can you add this plugin for 32 bit, also?
Is it possible to get source code?

32bit version is not added for simplicity and this project is not open source.

3 Likes

Can you please summarize if / how concurrency (e.g. thread and/or process) is supported on the python side?

It depends on what you are exactly interested in.

A single call to AmiPy's PyEvalFunction or PyLoadFromFile works one thread at a time.
It means that although AmiBroker would call them from multiple threads, it will serialize calls, so only one thread at a time is inside AmiPy plugin code. That is because calls to 3rd party plugins are protected by critical section inside AmiBroker.

On the other hand, several Python libraries use threading inside their code (for example Tensorflow and numpy), so you don't need to do anything to benefit from multiple cores.

Also Python has its own threading libraries, but this is tricky programming and you are still somewhat limited by Python GIL (Global Interpreter Lock).

Congratulations! I am extremely happy to see this. I have a few systems where I needed to use python code and I integrated them with AmiBroker as COM Objects. It is a very complicated process and has limitations in parameters passing. So, I am eager to rewrite these using this plugin.
My questions:

  1. Is this in beta or final release ?
  2. Is this free to use ?
  3. Will there be any support available ?

I wouldn't mind buying a license if this will become a commercial product. Though I could program in AmiBroker almost everything I can do in Python, I was looking for using complex code & libraries already available in Python, without going through the huge effort to port them to AB. Your AmiPy plug in is the ideal platform for such integration, and I want to thank you again for providing it to the AB developers community.

2 Likes

One more question:
Is the Python function array argument limited to BarCount size? It should be, as array in AFL have this size limitation.
I there a way to pass an arbitrary size array > BarCount to the Python function, similar to a numpy 1D array?

Thank you for your opinion and as for your questions:

  1. AmiPy is still in development stage
  2. Currently it is free to use. As for the future it is not set in stone. It depends how many people are interested.
  3. As for support of course i will fix any errors and try to answer questions here but generally it comes "as is" without extensive hand-holding help.

Python function array argument is limited to BarCount size and currently you can't pass arbitrary size array to Python. It could be done, but not using variable argument function (passing Matrix as argument).

2 Likes

Hi Kuba,

And thank you so much for the answer. The parameter passing limitation issue I had when I used a COM Python function was exactly related to the impossibility to pass an array size > BarCount. I needed to pass a covariance matrix to a portfolio optimizer written in Python. A covariance matrix is an N x N matrix, where N is the number of assets in your portfolio (i.e number of ETFs in my case). And because that was done on a monthly time frame, I had a very small BarCount, and had to artificially limit N (number of assets) so that N x N < BarCount. This excluded larger portfolio combinations that I could not backtest.
I hope you will be able to provide a matrix parameter to AmiPy Python functions and remove this limitation. That would be extremely helpful for using all kind of Python portfolio optimizers already in the public domain. Porting one to AmiBroker is quite a non-trivial, extensive, elaborate task, prone to errors, and AmiPy would resolve this issue nicely, elegantly without the need to do any porting hard work. Hope you will be able to provide this matrix argument feature to AmiPy.

Sorry, but it is non trivial, extensive, error prone task and other word constructions NOT because trying to port to AmiBroker but it is that way because of being too difficult for you to be achieved on your own. That's what you should have written instead.

Such portfolio optimizer thing you described is done within max. a week and quite easily in line with other programming solutions. I know it because I once asked myself "Hey you, could you do it?". So I answered to man in the mirror "Yes, sure" and just did.

So that fact of many others just shows that it got nothing to do with AmiBroker as programming environment but with your programming capabilities (within AmiBroker at least).

(Me just being tired of reading poor excuses a la "Oh, so difficult in Amibroker" while in fact AFL is such a fun language (and AB a fun app) to create things. And let's be honest you wouldn't be able to do it via other languages either if you struggle there already.)

1 Like

This is an unhelpful, irrelevant and uninformed comment. You contribute nothing to the topic at hand. I recommend you take your mind reading exercises elsewhere, if you have nothing to say. This is a thread about Python < > AB Integration. Your comment doesn't belong here.

4 Likes

Guys, let us be civil OK? Please stay on topic and be friendly.

2 Likes

It is not irrelevant because it is AmiBroker forum overall and comment was about AmiBroker's capabilities on its own (compared to Python). So in fact it is on topic.

I think that main merit of this plugin is just allowing to use existing Python libraries within AmiBroker with ease, almost without writing any code. Like for example Tensorflow (machine learning, neural networks, nlp).
Something like "quick hack" to try out idea without spending days on writing something from scratch. @fxshrat is right that everything can be written in AFL and AFL's performance is likely to be much better but some people simply don't have time or desire to do so.
So it is not bad to have alternatives and I understand that people may be interested in those alternatives.

12 Likes

Yes, and absolutely no problem with that!
What I meant was that if someone has no time/desire/ability to do without those alternatives then it got nothing to do with AB itself. Because it would take them time, desire, ... in any other environment/language also.
His sentence made it look like as if it was problem of AB/AFL to port something to it (without alternatives) in general. But it's not. That was all I was trying to correct. So @stefans take it easy.

1 Like

Hi, i got a not responding after click sends to analysis. screenshot attached

Capture2

This means that it got stuck inside AmiPy plugin.