Cannot generate python wrapper for eventcore.core.EventDriver using PyNIH #251
Comments
|
To be clear, the issue is not happened at compile or link time. The issue happened when you import the result binary library into Python, and then you see the issue undefined symbol error. |
|
I dig in further and I think I know what had happened. module x;
class Problem {
} //this will cause error. The way to fix it is add a function into this moduleCode Sample 2 to repro issue: void hack41() {}
struct S2 {
//this will have issue, because no method in this class
}
struct Problem {
void add(S2 d) {}
} |
|
After the fixing the original issue solved and now it cannot wrap below code. import eventcore.driver : EventDriverSignals;
EventDriverSignals hack31() {return EventDriverSignals.init;}Same compile and link is OK, but when load it into Python then this symbol cannot be found: |
|
Probably from vibed? |
|
So in summary we have more than one issue here: I have figured out the 2nd issue which is why we cannot wrap 3rd party library class eventcore.driver.EventDriverSignals. This is because in autowrap project file pynih/source/python/conv/python_to_d.d line 300, when wrap a function marked as import eventcore.driver : Handle, SignalStatus;
alias SignalCallback2 = void delegate(SignalListenID2, SignalStatus, int) @safe;
struct SignalListenID2 { mixin Handle!("signal", size_t, size_t.max); }
class EventDriverSignals2 {
void listen(SignalCallback2 on_signal) {}
void addRef(SignalListenID2 descriptor) {}
}It looks there are still other issues to wrap class |
|
Atila has sent me a patch for DMD to see if it can work around. I paste it here to keep a record. diff --git a/src/dmd/dtemplate.d b/src/dmd/dtemplate.d
index 619cc0611..7159e3855 100644
--- a/src/dmd/dtemplate.d
+++ b/src/dmd/dtemplate.d
@@ -6186,6 +6186,41 @@ extern (C++) class TemplateInstance : ScopeDsymbol
*/
final bool needsCodegen()
{
+ if(minst) return true;
+
+ if (!minst)
+ {
+ // If this is a speculative instantiation,
+ // 1. do codegen if ancestors really needs codegen.
+ // 2. become non-speculative if siblings are not speculative
+
+ TemplateInstance tnext = this.tnext;
+ TemplateInstance tinst = this.tinst;
+ // At first, disconnect chain first to prevent infinite recursion.
+ this.tnext = null;
+ this.tinst = null;
+
+ // Determine necessity of tinst before tnext.
+ if (tinst && tinst.needsCodegen())
+ {
+ minst = tinst.minst; // cache result
+ assert(minst);
+ //assert(minst.isRoot() || minst.rootImports());
+ return true;
+ }
+ if (tnext && (tnext.needsCodegen() || tnext.minst))
+ {
+ minst = tnext.minst; // cache result
+ assert(minst);
+ //return minst.isRoot() || minst.rootImports();
+ return true;
+ }
+
+ // Elide codegen because this is really speculative.
+ return false;
+ }
+
+
// Now -allInst is just for the backward compatibility.
if (global.params.allInst)
{
@@ -6281,7 +6316,7 @@ extern (C++) class TemplateInstance : ScopeDsymbol
return false;
}
- if (global.params.useUnitTests)
+ if (global.params.useUnitTests && false)
{
// Prefer instantiations from root modules, to maximize link-ability.
if (minst.isRoot()) |
|
Unfortunately this custom compiler will crash when compile PyNIH for vibe.core.net.TCPConnection class. The code which can repro this being checked on branch bug5_pynih_tcp_not_work https://git.kaleidic.io/SIL/plugins/stable/carbon-d/-/tree/bug5_pynih_tcp_not_work |
|
I think something is wrong that you are ending up wrapping eventcore at all |
|
Start with just wrapping a simple module with just structs. Make it work in python. Then wrap carbon client. It might be it tries recursively to wrap vibed types in which case remove mongo and redis vibe versions and I don't think vibe will be used at all. Getting auto wrsp to work for vibed is much harder problem |
|
For everyone's info. Atila has released a new version of Autowrap package into Dub. Together with that and the custom DMD compiler, now we are at least able to wrap some part of the code successful (which we cannot do in the past), this code is at branch autowrap-from-scratch. https://git.kaleidic.io/SIL/plugins/stable/carbon-d/-/tree/autowrap-from-scratch Though the part we really care which is CarbonClient, we are not there yet. |
|
Also I have a different branch tinyredis for Carbon-D project, which I have already migrated to use tinyredis and mongd. And when I compile it use the latest autowrap version and my custom DMD, I will run into an assertion error and then DMD core dumped. Below is part of the error message looks like. |
|
Here is the latest status:
|
|
I also spend more time on narrow down why compile CarbonClient will cause issue. There are at least two issues. alias Variable = from!"kaleidic.sil.lang.types".Variable;
export Variable hack51()
{
return Variable.init;
} |
|
Atila has pointed out that when we wrap CarbonClient, if we use dmd 2.092.0 then it will be core dump, but if we use 2.091.1 then it will be not. This is very helpful because then at least for now we do not need to depend on custom DMD compiler. And this is to complain below symbol: extern (C) nothrow python.raw._object* python.type.PythonBinaryOperator!(std.typecons.Nullable!(std.datetime.date.Date).Nullable, mirror.meta.traits.BinaryOperator("+", 1))._py_bin_func(python.raw._object*, python.raw._object*) |
|
I found that autowrap cannot wrap a quite simple struct BondEntry successful. I think this should help to identify why autowrap cannot wrap it. alias BondEntry = from!"symmetry.carbon.clienthelper".BondEntry;
And here is the definition of BondEntry:
struct BondEntry
{
import std.typecons : Nullable;
@Name("floaterFlag")
bool isFloater;
@Name("maturityDt")
Date maturityDate;
@Name("issueDt")
Date issueDate;
@Name("effectiveDt")
Date effectiveDate; // CHECKME
string isin;
@Name("ccy")
string currency;
@Name("cpnRate")
double couponRate;
@Name("firstCpnDt")
Nullable!Date firstCouponDate;
@Name("auctionDt")
Nullable!Date auctionDate;
@Name("mkt")
string market;
@Name("floaterFlag")
bool isLinker;
@Name("ric")
string ric;
}And I got into this symbol error: extern (C) nothrow python.raw._object* python.type.PythonBinaryOperator!(std.typecons.Nullable!(std.datetime.date.Date).Nullable, mirror.meta.traits.BinaryOperator("+", 1))._py_bin_func(python.raw._object*, python.raw._object*) |
|
I have just approved one thing. For wrapping CarbonClient class, if I mark all method which return kaleidic.sil.lang.types.Variable or BondEntry methods as package rather than public, I'm able to compile the code and load it into Python without symbol error. |
|
And for issue of BondEntry, I narrow it down further to Nullable!Date cannot be wrapped. Nullable and Date are all from Phobos library so this looks a autowrap bug. export struct Hack61
{
import std.typecons : Nullable;
import std.datetime : Date;
Date m1; //OK
Nullable!int m2; //OK
Nullable!Date m3; //not OK
} |
Autowrap cannot generate python wrapper for interface eventcore.core.EventDriver correctly using PyNIH. The client code will be like below to be wrapped:
And after compile, when load it in Python, an undefined symbol error message will be printed out:
The text was updated successfully, but these errors were encountered: