Gizmo / X-plane crash when sending datarefs

[Ben Russell]

Part of the problem fun is that Gizmo gives very raw access to the scripts running systems.

It drops right into IXEG systems with as little overhead or governance as possible.

Because of this it could be a couple of rounds before we really get this sorted out.

 

For now I'm going to focus on porting the replay system into the plugins realm.
This lets me lock out all the manipulators and commands from any scrambled triggering of state and systems while the replay scrubber gets dragged around.

 

I've had a naive look at this a long time ago. I don't own any hardware so I had to hack together a test harness... it's pretty basic.

If I could figure out a software harness that can replicate the same kind of crash as hardware-module owners are seeing we'll be a long way towards fixing this up.

https://github.com/benrussell/xpl_cmd_flood/blob/master/xpl_cmd_flood.c

[Ben Russell]

Also thinking about adding a "Hardware Mode" to Gizmo that slows it down somehow, perhaps caching the datarefs in lock step with the frame counter as a clock signal.

You might suffer a minor FPS hit because of the extra book keeping but it's a fair trade.

[Slayer]

That would be acceptable.

Goflight interface tool has delays and such but that doesn't help with any kind of sync issues.

What I don't understand is why the commands don't cause issues? Must be it's like an abstraction layer so "we" don't poke a dataref directly into the mix, instead it runs some kind of LUA macro or script which then access the datarefs?

[737NUT]

Thanks Ben, Please know i am in no way trying to knock Gizmo,  I just know that here are the facts in my own little world.

FlyJSim 732, using SASL, I have zero issues using my arduino based interfaces reading and writing to over 200 FJS specific Datarefs.  

I load the IXEG 733, instant 10-15FPS increase!!!  Awesome,  so yes Gizmo is efficient in that term. But, will not handle one of my interface cards without a crash.  So in my lack of knowledge I blame Gizmo.  I know it's not fair but its all I got.

Again thank you so much for putting the effort in to help the minority of users.

Rob

[737NUT]

14 hours ago, Ben Russell said:

Also thinking about adding a "Hardware Mode" to Gizmo that slows it down somehow, perhaps caching the datarefs in lock step with the frame counter as a clock signal.

You might suffer a minor FPS hit because of the extra book keeping but it's a fair trade.

I see a 3-4 FPS drop in the 732  that uses SASL when I turn on all my interfaces.  I'll accept that as normal and minor for allowing all the control I need.

Rob

[Slayer]

This problem has been solved pretty much.

The author of the program figured out a work around  "all reading, writing and commands are processed off the back of the widget draw event. "   (I'm not a programmer so don't ask how:))  was the fix, it may aid others in solving their issues using other software tools and methods to send datarefs to this aircraft.

After a few hours of testing I only had 1 crash because I accidentally send a out of range value.

Anyone else reading this that may use Goflight interface tool software will need to update to version 1.4 from your user account on PPS website.

I'll be releasing a profile with full MCP and EFIS support shortly.

[737NUT]

13 hours ago, Slayer said:

This problem has been solved pretty much.

The author of the program figured out a work around  "all reading, writing and commands are processed off the back of the widget draw event. "   (I'm not a programmer so don't ask how:))  was the fix, it may aid others in solving their issues using other software tools and methods to send datarefs to this aircraft.

After a few hours of testing I only had 1 crash because I accidentally send a out of range value.

Anyone else reading this that may use Goflight interface tool software will need to update to version 1.4 from your user account on PPS website.

I'll be releasing a profile with full MCP and EFIS support shortly.

When is the update going to be released?  1.0.3 still doesn't work for me. Gizmo pukes out in less than 2 mins with my hardware hooked up.

 

Thanks

Rob

[Ben Russell]

1 hour ago, 737NUT said:

When is the update going to be released?  1.0.3 still doesn't work for me. Gizmo pukes out in less than 2 mins with my hardware hooked up.

 

Thanks

Rob

Let's be clear here.

The problem lies with the third party plugins not fully understanding how the SDK hangs together.

 

If you can stop pinning this on me that'd be tops.

[mmerelles]

10 hours ago, 737NUT said:

When is the update going to be released?  1.0.3 still doesn't work for me. Gizmo pukes out in less than 2 mins with my hardware hooked up.

 

Thanks

Rob

GIT 1.0.4 fixes the issue, go and download it

[737NUT]

9 hours ago, Ben Russell said:

Let's be clear here.

The problem lies with the third party plugins not fully understanding how the SDK hangs together.

 

If you can stop pinning this on me that'd be tops.

Wow, ok, that's a turn around from your earlier post.  I am not a programmer, just a dumb pilot. But this dumb pilot knows one thing for sure. I have no problems with my 3rd party plugin and hardware with SASL based 3rd party aircraft in x-plane. Why? Don't know and don't care, it just works and that is what i want and need.  If Gizmo won't play nice with my dumb 3rd party plugins then so it be it.  It's a shame but hay, at least you got a 75.00 donation for building a very cool 733. 

15 minutes ago, mmerelles said:

GIT 1.0.4 fixes the issue, go and download it

Link?  Last I heard hotfixes were mailed to customers who purchased the 733.  Unless that post was sarcasm? In that case, guess i'll get a refund.

Regards,

Rob

 

[Slayer]

I was referring to Goflight interface tool. Version 1.4 corrected the issue for me, not v 1.04 of the IXEG.

Not sure what software you are using to send datarefs but it will probably need to be updated. 

[mmerelles]

14 minutes ago, 737NUT said:

Link?  Last I heard hotfixes were mailed to customers who purchased the 733.  Unless that post was sarcasm? In that case, guess i'll get a refund.

Regards,

Rob

 

GIT = goflight Interface Tool not the ixeg bird.

 

 

[737NUT]

I don't use goflight equipment but thanks for the heads up.  

[737NUT]

1 hour ago, Slayer said:

I was referring to Goflight interface tool. Version 1.4 corrected the issue for me, not v 1.04 of the IXEG.

Not sure what software you are using to send datarefs but it will probably need to be updated. 

Thanks,  I use Teensy hardware and the plugin designed for them.  I know what has to be done.  I have to get someone who knows how to program to change the plugin by sloooooowing it down so it doesn't send to much to fast.   Or I just give up and fly the stuff that works.  Bummer as I have quite the platform to fly the 733 on.    Gizmo for the win. 

 

Rob

[Slayer]

I'm afraid that slowing the events down will not correct the issue, slowing it down just makes it less likely that a crash will happen , but eventually it will unless the plugin is changed. It has to do with the programming method in which the datarefs are sent to X-Plane. 

Without saying too much that is over my head and I don't fully understand, they need to be sent with a widget draw event not just a plain draw event.

I'm not sure that this is completely fully documented in the XPlane SDK or why other aircraft from different vendors work without it being done this way , but it works for the IXEG/GIZMO. But yes you would need whoever made the Teensy plugin change it to send datarefs via that method.  

I'm sure someone with advanced programming skills in x-plane datarefs would be willing to help, try contacting the author of your plugin and direct him to this thread?

 

[Ben Russell]

Some parts of the SDK look like a "single call back entry point" when in fact these functions can be called repeatedly with different datarefs indicating a subtle yet important change of state.

This leads to naive creation of plugins that _appear_ to only query gizmo for data once when in fact they're probably querying for everything ~five(I'm not inclined to do an audit right now to find the exact numbers) times in the space of a single frame.

This leads to complications which manifest as a stack overflow.

There's nothing "wrong" inside Gizmo. It works just fine with X-P, DRE, DRT and now the GoFlight Interface tools from Pollypot.

The reason it explodes is because I chose optimisation and power over data safety, management and "hand holding"

These third party plugins are interfacing directly with the systems that the IXEG team have written and the third party plugins are doing a crap job of it.

 

The reason you don't see this with anyone else?

They probably use the concept of data buckets. Dataref value storage that's isolated from the core of their code. It's easier to write and far more resilient to stupid plugins blasting it with unsolicited overzealous requests.

There is no logic to run because the internal systems have already calculated all the values and moved them into dref containers.

There's also no branching into code that relies on the underlying JIT engine with dynamic memory allocation and all the fun that brings.

 

It's complicated. I have a good idea of what triggers the issue and it's not something I need to fix... or probably even can fix now that we have multiple premium products built on this.

Any attempt to cache value or somehow be "smart" about it will undermine the point - and power - of Gizmo.

There are legitimate reasons to change a dataref value 5 times in what appears to be only one graphics call back.

 

In short;

You're going to need to get your hardware interface plugins updated one way or another.

Given a few more weeks (as in probably more than a month) .. I might have the time to go and patch someone elses open source plugin. 

Right now we need to serve the majority.

Edited May 17, 2016 by Ben Russell
Low Caffeine.

[Slayer]

Ben,

Thanks for the explanation. 

DRT's method for access works and it is open source.( and a clue on how to make it work) for other hardware  plugins and I'll leave it at that.

I'm just happy I've  got my hardware working no matter how the result was achieved.

This thread has probably run it's course and you can close it if you want, there was no intent of finger pointing of blame laying, hope you don't take it the wrong way.

I was just providing my findings  to help other people get their hardware working too. Your explanations of how things worked were helpful.

Thanks

[Ben Russell]

I'm glad the extra detail helps.

I figure the best I can do at the moment is try and share the insights.

If it's all "geek"; well, at least it shows that I want to be open about the issue at hand and any methods we can use to resolve issues.

 

Back to the Arduino side of things;

It's far more accessible to me on every level to fix Arduino stuff than any other hardware platform.

- Costs less.

- Uses less space.

- Repurposable.

 

I will get to it. I have a box of Arduino hardware already.(2x Uno, 1x Mega + random sensors)  It's just a matter of working through the queue.

 

Edited May 17, 2016 by Ben Russell