Auto weight unbalanced binary output values in residual / sum of square error computation (for classification problems) based on the actual data imbalance ratio. For example, if you have 25 outputs at 1 and 75 outputs at 0 in a 100 point data set, the imbalance ratio is 3 to 1, so we weight the errors on the actual 1 values to be more important by a factor 3, so that the solver will focus on those better. This factor may be high, but we leave it there for now. Allow the "max down count" that can occur before stopping is triggered during a fwd stepwise solve to be 10 (was 2). Output the so-called precision of a binary outcome solve for the 1 target value during a stepwise solve. The following right hand side variables are integer counts. float mlPrecision = (float)correctBinary1/((float)(correctBinary1 + (actualBinary0 - correctBinary0))); Is this the only definition possible for "precision?" Doubtful. This is why we output the raw ratios (confusion matrix) for further analysis.

Additional performance improvements for forward stepwise method. Random-seed the model creation earlier in the process before initial randomized basis function placement to achieve consistent results run-to-run.

Change basis function looping to loop only over unmasked basis functions so that forward stepwise solve is faster, for point evaluation and design matrix generation. Reduce screen report outs to 1 per second for speed. Don't route point evaluation blocks to queues when ncpu is set to 1 to avoid queue overhead. Compiler optimizer on for speed.

Runtime speedups.

Reduce width jittering during optimization (reduce width-specific "learning rate") since the model is very sensitive to function widths. Switch to a gradient descent optimization approach for the "mc" setting (with monte carlo aspects, so leave the UI to say "mc"). Features of new optimization changes: dynamic learning rate: • if we are learning well, increase learning rate by 1% each step • if we are not learning well (no new best solution found), reduce learning rate by 1% at each step to some lower limit close to zero (currently we do this only after 5 or more failures to find a new best point, to give some tolerance for failures) When estimating a gradient at the current best point, use a stochastic method due to our code base that is based upon random perturbations of a point: P) perturb the current point not via a constant delta X in each direction but via a random small amount in each direction (different random number close to zero in each X direction) [note that this method also avoids us having to compute nparameters-to-optimize number of partial derivatives] • as the main loop iterates, we reduce this random perturbation for gradient estimate linearly so it is larger at loop start and smaller at loop end, "zooming in" on a solution repeat P) above, up to 5 times: for the first of these 5 that yield a better point (if any), choose this point and do not compute the remainder of the 5 (early terminate the gradient estimation loop) end repeat if the found new best point is in the correct direction (improved result), mark that this is a good gradient estimate, and extend the search in that X direction according to the dynamic learning rate noted above if there was no found new best point, take the last new point we found (in the wrong direction) and move the next point selection in the Opposite X direction of it if this new point is now "best," save it if not, go back to the prior best point before going to the next step local min bounce out feature every 10% of the main loop iterations (randomly chosen, not "every 10") bump the "gradient" estimate point much farther than we ordinarily do, to try to bump the optimizer out of a local minima in case it encounters one. For now, this extra bump is fixed 100x the gradient delta we would ordinarily use at that step.

Fix problem: after pressing Stop button on larger data sets, the code would get stuck in a 'trying to plot too many points' section of the code.

Add periodic prompts at app launch to direct user to related articles and case studies.

Add a feature to allow the user to specify direct random number generation during the solve process. The random generator is seeded with a constant so that repeated solves of the same problem give the same answer. The prior cached random mode, which is faster in some instances, remains available.

Stop the generation of two random seeming dot plots when mode is set to MC only (monte carlo optimizer only). These should only be displayed when in the matrix + monte carlo solve mode or the matrix only mode. Remove early terminate of monte carlo solve for now, this needs more tuning. That is, run all monte carlo optimizer solves to the user specified iteration end. Change the output folder locator button to be a folder export feature to comply with Apple policies. The output folder can now be saved on your macOS drive home dir or other folder location in your personal file space. Each exported folder is given a unique date and time stamp.

On first app run, trigger the "reset" parameters feature so that the initial install has the same default parameters that the reset button provides.

Speed up fwd stepwise solver.

Add button to reset parameters to default.

Allow user to toggle verbose / silent for controlling output text.

More verbose output for analysis of solves. For forward stepwise, use linear fit method for the stepwise portion of the solve if "mat + mc" method is specified. Larger fields for file name display on upper right of app screen.

Speed up input CSV file processing. Save & restore max cpu core count to / from user defaults.

Allow parallel operation of the design matrix generation, especially during the forward stepwise solve method, using the same max CPU core count as specified on the user interface. This allows faster stepwise solves for larger problems. It seems useful to set the CPU count number slightly higher than your target CPU core use for faster run time.

Don't make actual/predicted plots if more than 5k points, for speed-up on larger data sets. User can use the output folder button to find the actual versus predicted data files and plot them separately if desired, in other tools.

Indeterminate mid-zone tolerance for binary (categorical) output forecasts is now user-settable. Provide info button for this to remind users what it is for. Minor UI update for clear/trashcan button so it is easier to see.

Allow the target y variable to be specified as categorical (binary 0, 1) in the user interface. In previous versions, the categorical setting for the user-selected y column did not influence the solver metrics report-outs. Now it does. Only 0 and 1 y data values are supported now (not, for example, -1 and 1). Output y column must be numeric, no quotes allowed in the data. When you flip on the categorical switch for the y values, metrics for binary classification are output when solving, along with traditional R2 etc. Aggregate correct and fraction correct are reported, along with metrics for the individual output classes / categories (0 and 1). Example output lines for categorical output: R2 = 0.7400 RMS err = 0.028134 maxe = 0.770282 mine = -1.003435 AdjR2 = 0.7396 TRAIN of n = 38324 points, mat solve aft bkwd1, nbasis = 50 [linear = 22, dir = 28] correct 38299/38324 = 0.999348 indet = 0 1: 103/117 = 0.880342 0: 38196/38207 = 0.999712 The second line of report-out is as follows: correct (aggregate metrics correct/all) indet (indeterminate forecast count) 1: (correct/all metrics for category 1 output) 0: (correct/all metrics for category 0 output) In the above example, there are far fewer outputs with y data value 1 (count of 117)-- showing 103 of them forecast correctly (103/117)-- than of data value 0. An indeterminate grey zone of +-0.01
(hard coded) is specified to filter out indeterminate values. E.g. if a predicted value is within [0.49, 0.51], we treat it as an indeterminate result, not to be categorized as a 0 or a 1. The count of these indeterminate results are reported with the "indet" annotation (see above). This grey zone will be tunable in an upcoming release, as we found in other studies that this tuning can have useful benefits for forecasting. We recommend the settings [fwd+] (forward stepwise) with [matrix] solve method to be used for binary models to start. Also, elapsed time per each "best forward step" is reported out.

Allow forward stepwise solver method with monte carlo stochastic optimizer per step as a new option. This allows basis function width and epsilon shape factor to come into play during fwd stepwise regression. For all uses of the stochastic optimizer, early terminate the optimizer loop if improvement gets slow. Slow improvement criteria is: change in SSE less than 0.1% more than 2 iterations in a row.

Provide an (opt) button to optimize global basis function width 10 steps at a time using a stochastic optimizer. You can re-run the (opt) to try to get better results, and it will continue to optimize from where it had left off. If you have a test point count set up to be greater than 0, it will try to optimize the test point forecast metrics (e.g. reduce RMS error of test set). If you have test point count set to 0, it will optimize to training set RMS. A popup reminder of this is provided when running the option. Also update main app window so that it doesn't need a main scroll view if you are running on a larger screen. This was only required for smaller screen support.

Fix ridge optimizer. Allow better app usage on smaller screens by allowing user to scroll the main app window.

Caution the user if he has used up all of his data for test points with none left for training the model. Don't flip the solve mode back to monte carlo optimization every app launch. Leave it where it was set.

Allow X dimension single direction basis functions as an option, by themselves in a model or along with the radial-like and linear functions already available. Separate candidate functions are generated during the random basis function center point selection process for each direction of each X variable, e.g. there are 2 separate directional basis functions generated for x1 (towards the positive x1 axis and towards the negative x1 axis, and so on for variable x2 etc). Control over this is via the new [rad | dir | &] segmented control button. Use the '&' setting to search thru both directional and radial functions when building the model. Separate +x and -x functions are made so that the solver can pick one or the other or both of them as needed to model the data best. When a center point is chosen for a basis function (currently by random selection of the training points), directional functions for all variables (both + and -) are generated as candidate functions, even if we overflow somewhat the user set max basis function count, for dimensional completeness. So if we have 4 X variables, there would be 8 candidate directional functions generated at a point, even if we had set max basis functions to 5. During the forward selection solve process, the functions that decrease the SSE (equivalently RMS) error the most are chosen first, as is done for the other basis functions in this system. The directional functions are searched before the radial functions if both are requested. The y value of the half of the basis function that is not chosen is set to a constant y value; in the current setup, y = 1. In the case of the SimplML system now, the non-linear basis functions are always 1.0 at radius zero, so this 1.0 is carried out to make the function flat in the non-chosen direction. In our case, basis function choices are gaussian decay (starting at 1.0 at 0.0 radius with the function value decaying in an (a) gaussian manner toward zero, that is: decaying slowly, then quickly, then slowly again); or a (b) multiquadric smooth ramp-up away from radius zero, yet flat (zero derivative) at exactly zero. If the epsilon shape factor is set large (say, 10.0 or larger), the multiquadric approaches asymptotically a linear ramp shape with a slightly rounded corner at 0. Adjusting epsilon allows the ramp corner shape to be tuned to give smoother (or, alternately, sharper cornered) model surfaces as the analyst sees fit. This can be visualized in our Excel chart example by pressing the example button to the right of the epsilon text field in the app. The effect of the epsilon corner shape adjustment can be easily seen if you fit data with alternately high and low values of epsilon (all else held equal) and then use the slice tool in the app to cut through the 2 models to view the cross sections. To achieve an exhaustive search over these new directional functions, set the start basis count to be greater than the number of points in your training set, and set 'max fract basis [0,1]' to 1. Note also that SimpML's Adjusted R2 counts all basis functions equally at the moment for penalty purposes, even though the radial functions can account for non-additive variable interactions in the data, and the directional and linear functions cannot. In the green output window, the system reports out how many directional functions it chose during the forward stepwise process, along with total basis function count and linear function count. Small adjustments have been made to the annotations of these counts.

Report out how many linear terms were used in the model in the green output area. Suggest that user set the test point count to 0 for ridge optimization. Fix issue with eval points feature not saving file. Add plot for evaluated points.