Forum

snovotill · Joined: Feb 8, 2013 Posts: 80

Hello Jarek. I'm having some trouble understanding scale factor handling and internal representation of numbers. so post here thinking it will be a good general reference. With the understanding that BioEra uses the 32 bit signed integer range internally ie 2147483647 through -2147483648 here are some puzzles:

MIN & MAX: In Advanced Signal Properties, if I enter a value in excess of 32 bit signed integers, then this gets replaced by MIN and MAX. However I have confirmed that MIN is actually NOT equal to -2147483648, and that MAX is NOT equal to 2147483647 which surprises me. So I wonder what MIN and MAX are normally supposed to be used for? It seems to me that MIN and MAX are context sensitive rather than statically defined.

SLIDER: When I enter into Slider properties MinimumValue -88388608 and MaximumValue 8388607 then I get incorrect output values, and slider only allows 2 positions to be set(I was experimenting with 24bit A/D values). if I enter the full available range 2147483647 to -2147483648 then again similar results. I did try to adjust Advanced Signal Parameters but of course those should only affect Slider's inputs. I'm guessing that most of the math in BioEra is not 32 bits, and wonder what a safe min and max signal level would be in terms of best-practices.

NUMERIC-DISPLAY: For NumericDisplay I enter DigitalMin -2147483647 and DigitalMax 2147483646 and PhysicalMin -1 and PhysicalMax 1. It doesn't like me at all, and I'm getting overflow readings even with \"normal\" input values, yet the above values fall withing 32 signed integer range. What does this mean and what are the constraints to be adhered to?

We know that 1uV=20%=1000digi at low signal levels, but what about small uV signals riding on large DC offsets, as may be seen in \"balanced uV\" or \"balanced digi\" scenarios? What are the numeric constraints that one should take into account in light of the above? It kind of looks like there may be a scaling factor of 1000 built into BioEra which is supplementary to the main scaling factor created via Advanced Signal Parameters.

RESCALE-INPUTS: Expression Evaluator has the Rescale-Inputs option which I'm confused about. If I turn that OFF then Expression Evaluator rescales it's inputs according to whatever I enter into Advanced Signal Parameters. If however I turn Rescale-Inputs on, then Expression Evaluator ignores all Advanced Signal Parameters completely, and does a rescale of inputs based upon some scaling factor which it gets from somewhere else??? I can't figure out what it's doing in the latter scenario, or where it's getting it's special scale factor from.

DOWNSTREAM SCALE FACTORS: When I enter scale factors via Advanced Signal Parameters for all elements which are chained in series, BioEra seems to ONLY use the scale factor of the last element in the chain, and changing the Scale Factors in upstream elements makes no difference to the final signal. I would have thunk the changes would be cumulative. Is this always the case, or are there exceptions?

SCALE FACTOR PROPAGATION: Sometimes scale factors propagate forward (or backward?) through the chained elements. Yet if I set manual scale factors on an upstream element, those seem to stay with the element and do not propagate anywhere; downstream elements keep their own default scale factors. Scale factors from Slider and Signal Sources seem to propagate all over the place. How does this work? Is there conscious control over it somewhere?

DEFAULT SCALE FACTORS: The scale factors created by default when placing a new element, seem to often have Min=-1 and Max=1. Are these to be taken literally? Why not MIN and MAX or some other value? Does the absolute magnitude of these numbers make any difference to the internal representation and how the values are transmitted between elements?

Farther to above, we have:
Output=Input*(PhysicalMax-PhysicalMin)/(DigitalMax-DigitalMin)
But why specify both a min and a max. I noticed they are mapped symmetrically along zero ie Physical 0 is always Digital 0, so In order to get the ratio we only really need to specify Digital Max and Physical Max. There is a good reason for supplying the Min values, right?

IMPORTANT NUMBERS: When I feed 1932735 into NumericDisplay(float4) via Property then I get a positive number displayed. If I feed in 1932736 via Property then NumericDisplay overflows the sign bit and goes negative. While this isn't unexpected, it's surprising that it isn't happening around 2147483... mark.

best,
...Stepan

jarek · Joined: Oct 22, 2007 Posts: 1073

If the digital ranges are so confusing perhaps it is better to use float elements? The float (and double float) elements were created for exactly that reason - to avoid problems and confusion with scaling.

Yes, the integer is 32 bit signed number. But please keep in mind that operations (like multiplication) are performed on integers so they can overflow in some cases.

In the past I only allowed up to 16bit integer ranges. That way if two 16 bit integers are multiplied, they are still within 32 bit number.

Currently most elements like ScalarTimeTransform can handle more than 16bit, there is more and more devices with 24 bit output. But perhaps not all of them, Slider might be such case.

So in summary, try to stay up to 16bit numbers, or use float elements.

This is the correct formula:

Output=Input * PhysicalMax / DigitalMax

All signals should be either symmetrical (-N..+N) or start from 0 (0..+N). Other ranges are not supported (even though you can force them in advanced properties).

snovotill · Joined: Feb 8, 2013 Posts: 80

That actually provided a lot of good perspective Jarek. Here are my notes on Scalar Values for everyone to enjoy, along with a few remaining questions:

INTERNAL REPRESENTATION:
BioEra uses 32 bit signed integers internally ie 2147483647 through -2147483648, and these are represented by the built in constants MAX and MIN. Unsigned 32 bit integers in the range 0 through 4294967295 are not permitted, although NumericDisplay is able to display numbers in that range AFTER it applies scale factors. Scale Factor 1000 is optimal for performance to do all calculations on 32 bit integers; setting Digital Max to be 1000 times larger than Physical Max in Advanced Signal Properties yields real numbers with 3.000 significant digits of precision behind the decimal point.

BUILT IN UNITS:
We have the following built in relationships between the available units:
1uV=20%=1000digi, 1%=0.05uV=50digi, 1digi=1nV=0.02%
However scale factors are only applied to signals defined in uV, having
have no effect on signals with unit % or digi.

BEST PRACTICES:
Keep signals to be processed within 16 bits aka 32767 through -32768 to ensure that multiplication results fit into 32 bits. Some elements can output up to 24 bits, for instance ScalarTimeTransform handles more than 16 bits.
##### QUESTION: Are unsigned 16 bit values 0 thru 65535 considered safe?
##### QUESTION: What is the permissible range for 24 bit capable elements?

ADVANCED SIGNAL PROPERTIES:
All Physical and Digital values should be either symmetrical (-N same as +N), or start from 0 (0 through +N) or (0 through -N), in which case we have Output=Input*PhysicalMax/DigitalMax.
Other ranges are not supported even though you can force them in Advanced Properties, and in that case we have Output=Input*(PhysicalMax-PhysicalMin)/(DigitalMax-DigitalMin).
Setting unsupported asymmetrical ranges via Min & Max value pairs does not create any signal offsets, so Physical 0 is always at the same time also Logical 0.
Span between Min & Max values must not exceed 21474836 so at the extremes are:
Min=-10737418 & Max=10737418, or Min=0 & Max=21474836, or Min=21474836 & Max=0
##### Question: Does the absolute magnitude of these numbers make any difference to the internal representation and how the values are transmitted between elements, given that the ratio between them remains the same?

SLIDER CAPABILITIES:
Slider will work within the following extreme min & max values:
Min=0 Max=2147483.8 (yields 2147483647 internally)
Min=-2147483.5 Max=0 (yields -2147483520 internally)
Min=-1073741.8 Max=1073741.8 (-1073741760 & 1073741760)
So the span between min & max values must not exceed 2147483.6
Slider is able to generate the value MAX but not the value MIN.

NUMERIC DISPLAY CAPABILITIES:
NumericDisplay seems to be able to display the following ranges:
Digi Decimal: -2147483520 thru 2147483647
uV Decimal: -2147483520 thru 2147483647
uV Float1: -214748364.7 thru 214748352.0
uV Float2: -21474836.47 thru 21474835.20
uV Float4: -214748.3520 thru 214748.3520
There might be some loss in precision behind the decimal point.
Larger numbers can be displayed only if magnified by a scale factor.
NumericDisplay is able to display the value MAX but not the value MIN.

ELEMENTS AND SCALE FACTORS:
ScaleFactors set in Advanced Properties only apply to that element's own inputs, and the values set therein do not propagate to downstream elements like they would from Slider or a SimulationSource.
##### QUESTION: Under what circumstances do scale factors propagate downstream as for instance from SimulationSource?

RESCALE INPUTS:
Rescale Inputs is available in Threshold, ExpressionEvaluator, BarDisplay and possibly others. Turning this on forces the scale factors set in Advanced Properties to be ignored.
##### QUESTION: Rescale Inputs seems to apply scale factors from upstream but this requires farther clarification or experimentation.

jarek · Joined: Oct 22, 2007 Posts: 1073

##### QUESTION: Are unsigned 16 bit values 0 thru 65535 considered safe?

Yes

##### QUESTION: What is the permissible range for 24 bit capable elements?

30 bits.

##### Question: Does the absolute magnitude of these numbers make any difference to the internal representation and how the values are transmitted between elements, given that the ratio between them remains the same?

The same ratio may not be enough. For example Polygraph requires the same ranges on all inputs.

##### QUESTION: Under what circumstances do scale factors propagate downstream as for instance from SimulationSource?

It should always propagate. But if you want to see the signal properties which are propagated, connect a neutral element (a one which doesn't modify SP) like Debugger and see it there (on its advanced properties). This is a general rule. Because if you open advanced properties on Simulator, you only see values which you set manually there, but not those which are set by the element itself (in this case rate is set in the main properties, but some elements like Decimator also change rate).

Please note, that values which element sets internally take precedence (for example if an element sets rate, then there will be no use to set it on advanced properties of the same element, you can do this on a downstream element).

##### QUESTION: Rescale Inputs seems to apply scale factors from upstream but this requires farther clarification or experimentation.

Yes they do.

-----------------------

I think it might be really a good idea to consider Float elements, they are still behind and perhaps not as well tested, but most should be fine. I personally use them almost exclusively. And they can be mixed with Scalars (if needed).

snovotill · Joined: Feb 8, 2013 Posts: 80

Jarek, I can't thank you enough for the clarity you've given to my understanding of Scaling. I feel like I've really got it nailed down and hope others can benefit from this excellent discussion.

As for the use of floaters, my designs which seem to have grown to hundreds of elements are already killing my CPU despite down-sampling wherever possible and use of scalars... but I'm surely looking forward to the simplicity of floaters after I buy a real computer next year.

snovotill · Joined: Feb 8, 2013 Posts: 80

Jarek, there's one more scale factor thing I cannot figure out. By way of example:

I've got Slider going into RangeMapper going into Generator, for setting the frequency.
Range mapper outputs digi values ranging from 150000 through 900000, for 150Hz through 900Hz.
Advanced Signal Properties of RangeMapper are Digital Min=0 Max=1000 and Physical Min=0 Max=1.
Debugger sees the correct values going into Generator, for example 240000 for 240Hz.

But Generator is going nuts because the wire going between Mapper and Generator
is stuck on really bizarre scale factor values, instead of the values which I expressly specified
to use, within Advanced Signal Properties of Mapper and also in Advanced Properties of Generator.

I cannot figure out how to override these strange values in the wire, to tell it that scale factor is 1000.

NOW: I've been using two cascaded expression evaluators which both contain the expression \"In1\" to
fix this issue, but that seems like a very convoluted way of fixing such a simple and common scalefactor mismatch.

snovotill · Joined: Feb 8, 2013 Posts: 80

Farther to above... I just noticed this: It seems that RangeMapper is the culprit. It seems to be ignoring whatever I put into it's Advanced Signal Properties. If I put any other element in between RangeMapper and Generator then I'm able to set scale factors in this intermediate element and it satisfies Generator and everything works.

This may be related to the post farther up above: \"This is a general rule. Because if you open advanced properties on Simulator, you only see values which you set manually there, but not those which are set by the element itself \"??? If this is so, then shouldn't such properties be grayed out? Otherwise it's very confusing and one cannot know whether they are actually setting a parameter, or just blowing smoke. Better yet if manually set scalefactors would always override everything else?

Also, I noticed that if I reload the entire design then the Signal Properties on the wire fall into sync with one of the elements it's attached to. It seems that it's necessary to reload the design in order to update/propagate scale factors to the wires and to upstream/downstream.

jarek · Joined: Oct 22, 2007 Posts: 1073

> If this is so, then shouldn't such properties be grayed out?

Technically yes, but it would be too difficult to implement. The advanced signal properties are generic, they behave the same for all elements. There is no generic way to determine which element sets what properties. I often recommend to set them only only on some elements which do not set them internally (e.g. ExpressionEvaluator or Valve).

snovotill · Joined: Feb 8, 2013 Posts: 80

Okay, so now all my scaling problems are gone and everything works perfectly everywhere.
These are my takeaways from the tail of this thread, in case someone else might be reading it:

(1) Scale factors set in an element do nothing more than propagate downstream with the signal, except if you have Rescale Inputs selected (where available), in which case the incoming signal is transformed to match the new scale factors, before being passed downstream. I have not found any instances(yet) where the scale factors would have any effect on the incoming signal; for that I've always needed to adjust the scale factors in the element immediately upstream.

(2) In order to completely assure that the design will work as expected after setting manual scale factors via Advanced Signal Properties, one must re-load the design so that it can \"re-compile\" and re-propagate the new SP to wires and downstream elements. Only then can you be assured you are seeing the correct parameters on the interconnecting wire right-click Signal Properties, and downstream in element Advanced Signal Properties.

(3) The elements which set their own scale factors own you and so you cannot override them, but you can put ExpressionEvaluator or ScalarValue or Valve downstream, and do your overrides in this \"inert\" element instead. I found this very useful because if I disable Rescale Inputs then I can change the resolution aka number of decimal places to the right of the decimal point in my signal(can be good after taking a logarithm), or I can implicitly multiply or divide (scale) my data stream by a constant(good to gain resolution when setting Generator frequencies).

I've got to get out of here now ;o)