05103: Support conditional piping/tailoring and complex calculations via embedded equation parser

ID	Project	Category	View Status	Date Submitted	Last Update

05103	User patches	Other	public	2011-04-11 20:37	2012-06-21 13:22

Reporter	TMSWhite	Assigned To	TMSWhite
Priority	normal	Severity	minor
Status	closed	Resolution	fixed
Product Version	1.91RC5
Target Version		Fixed in Version	2.00

Summary	05103: Support conditional piping/tailoring and complex calculations via embedded equation parser
Description	I'd like to add an embedded equation parser to LimeSurvey similar to the one I had to create for a different survey platform. Steps include: (1) Agree upon a way to delimit portions of a string that should be treated as plain text and those that should be run through the equation parser. In my case, I surrounded computable sections with back-tiks (`). For sake of discusion here, I'll assume we decide to use back-tiks, but I'm certainly open to using a different set of delimiters. (2) Create way to lookup Question-Objects and their associated attributes and answers by the variable name associated with the question. This way users can write complex equations using the variable names they are used to (as an alternative to having to figure out what the SGQA-equivalent would be). Since there is already a way to lookup questions by Question Number {Question: (3) Agree upon a grammar for the equation parser. Using JavaScript grammar is most convenient, because it is well known/documented, it can be debugged using existing IDE tools and FireBug, and there are published YACC grammars in case we need a compiler-compiler. (4) Agree on syntax for referring to Question attributes (in a read-only manner). For example, say you have a variable called BothersomenessOfPsychoticSymptoms (BOPS), and the question text is "{FirstName}, you mentioned having {X} psychotic symptoms, including {list-of-symptoms}. On a scale of 1-5, from not-at-all to severe, how much do they bother you on a daily basis?". And, say you've translated this into 15 languages. Some of the attributes of the question you might want to access include (but are not limited to): BOPS.question (to get text of question as rendered to user in whatever language they saw it in), BOPS.answer (to get the text of the answer choice they selected, such as "moderately"), BOPS.answerCode (to get the numerical value of the answer they selected, such as 3), BOPS.questionSrc (to get the unparsed text of the question - e.g. for debugging purposes). We might also want the variable name without dot extension to refer to the answer (since that is the most common usage). (5) Agree upon an execution engine for the syntax. Other groups, like the National Library of Medicine and Project REDCap both do the parsing directly in JavaScript, so they found ways to make that secure. I happened to write a JavaCC compiler-compiler for my system, but I don't recommend going to that extreme. Rather, if we can validate that there are no security risks in the equation, we can let JavaScript do the processing itself. This would make it easier to add additional functions and functionality. Say we agreed to use back-tiks as the delimiter, you might have the following questions, in comma-delimited format (variableName, Question, AnswerType, AnswerChoices). For AnswerChoices, the syntax is <message>,<code>[\|<message>,<code>]* hasChild, "Do you have any chidren?", list, "No,0\|Yes,1" male, "What gender `(hasChild==1)?'is your oldest child':'might you want your first child to be'`?", list, "Female,0\|Male,1" name, "What `(hasChild==1)?'is':'might you want'` `(male==1)?'his':'her'` name `(hasChild==1)?'':'to be'`?, text The parser would take the following steps for each question: (1) Divide the string into sections - those outside of the back-tiks and those within them, putting the collection of sections into an array. (2) Run each section within back-tiks through the equation parser, then concatenate the array sections back into a string as the value to be shown to the user. (3) For each section within back-tiks: (a) optionally use preg_match to find everything that looks like a function declaration and ensure it is a supported function (e.g. to optionally support only a subset of Javascript functions) (b) optionally use preg_match to find any other JavaScript syntax we're worried about and return an error message showing the requested function and the string position of the start of the detected problem. (c) At development-time, use preg_match to find everthing that looks like a variable (e.g. name of a question), and check against $_SESSION['fieldarray'] to ensure that each those variables exist and have already been "declared" prior to being used within the current question. If not, generate a list of variables which are not yet defined and return them to the user. (d) At run-time, use preg_replace to replace all of the variables names with JavaScript code needed to get the current value for that questions (or other specified attributes, like the question text or answer code). If despite step (c) the question hasn't been asked before the user gets to it (this can happen if there are conditions that make dependent questions not-applicable), then display the result as UNASKED Steps (1),(2),and (3)(a)-(c) would need to be applied when a user saves a Question or Answer during development Steps (1),(2), and (3)(d) would need to be applied at each place where a string to display is returned (e.g. qanda.php:answer_replace() - I haven't looked at all the locations yet). Once there is a mechanism to do arbitrary calculations, it can also be used for advanced validations and conditions to determine whether a question is relevant. For example, in some of the epidemiological surveys we've run, we first determine whether the patients meets criteria for depression (5 of 12 symptoms in last 2 weeks with no systemic cause), and then ask symptom-specific follow-up questions. We've had questions and groups with a dozen pre-conditions. I would recommend treating this sort of functionality as a power-user-only set of features. Although it might eventually be possible to create a GUI for it, I found the the epidemiologists I worked with preferred not to use a GUI, since they could have thousands of questions and it was easier to put them in Excel and use Excel-style filtering and searches to find the variables they needed and their allowable value-sets.
Tags	No tags attached.

Complete LimeSurvey version number (& build)	9992

related to	05288	closed	TMSWhite	User patches	Optionally replace Assessments with ExpressionManager features
related to	05269	closed	TMSWhite	User patches	Use ExpressionManager for Branching logic as optional alternative to Conditions
related to	05268	closed	c_schmitz	User patches	Do all LimeReplacementField and Token replacements in a single function
related to	05104	closed	TMSWhite	User patches	Create new question type for stored calculation results, called Equation
related to	05279	acknowledged		Development	Add a GUI for ExpressionManager
related to	04424	closed	TMSWhite	User patches	New URL field {GATE} to switch exit-URL and parameters by logical expressions

TMSWhite 2011-04-19 20:36 reporter ~14849	This proposal is a more generic version of Issue 04424: "New URL field {GATE} to switch exit-URL and parameters by logical expressions". Given the security risk of using eval(), I've been looking into some options to address this. The most robust solution is to build a compiler compiler with the stripped down functionality needed for processing these expressions. In general, I've found compiler compilers are easy to build and maintain in YACC, Bison, and JavaCC. However, they emit code for C, C, and Java, respectively. Unless LimeSurvey is open to having a binary plug-in (e.g. compiled C code), I presume you'd want a JavaScript and/or PHP-based compiler-compiler. Oddly, I've been having a hard time finding operational and maintained compiler-compilers for either JavaScript or PHP. Here is the best of what I've found so far: JAVASCRIPT: (1) http://silentmatt.com/javascript-expression-evaluator/, and https://github.com/silentmatt/js-expression-eval/blob/master/parser.js -894 lines of JavaScript, and clearly-enough written that would be easy to extend. However, it appears based upon a Shunting Yard algorithm rather than a Backus–Naur Form (BNF) grammar, so might be harder to add additional syntactic functionality (like array references) if truly needed. (2) RPA Search: http://www.rpasearch.com/web/en/parsejavascript.php Does use BNF, and has operational demo 95K binary (for windows, linux), GNU v3 license (3) Narcissus: https://github.com/mozilla/narcissus/ and http://mxr.mozilla.org/mozilla/source/js/narcissus/jsparse.js Uses a metacircular interpreter, which is harder to develop and maintain than BNF Requires installation of SpiderMonkey shell Not clear that actively maintained (4) FireBug: http://getfirebug.com/javascript This is a robust platform for JavaScript, CSS, etc. It includes equation building and debugging capabilities. However, it is very over-inclusive. Would need to check with developers to see whether there is a way to get a version that only does the needed JavaScript expression parsing. PHP: I could not find any operational PHP compiler-compilers. However, something like RPA-Search could do the trick

Mazi 2011-05-27 13:42 partner ~15135	Carsten, I assigned the ticket to you because TMS wanted to talk about some details and post further suggestion here. This way you will be notified.

TMSWhite 2011-05-27 15:53 reporter ~15136	Here's where I am so far, (using the numbering from above). I have proposed solutions for #s 1 and 2, and a comment/question about #3. STATUS UPDATE: (1) Syntax: I plan to use {EVAL: ... }, and treat the ... as an equation to be parsed where variable names are of the form question.title (2) Retrieve value by queestion.title - successfully prototyped that I could do this using {ANS: ... } via .\classes\dTexts\dFunctions\dFunctionAns.php (3) Grammar - this will be tricky. I haven't been able to find a decent compiler compiler for PHP, and although I wrote a stack-based equation parser in C 20 years ago, I don't want to recreate such work when there really should be a BNF or EBNF compiler compiler out there somewhere. QUESTION: (3) Grammar - In lieu of building a custom equation parser (although I'm happy to do this if someone finds a robust and well-maintained BNF parser that generates PHP), I'd like to find a safe way to validate a self-entered equation so that it can be parsed by PHP eval(). There are several pretty good examples of "safe_eval" here: http://php.net/manual/en/function.eval.php . I envision taking this approach: (a) Blacklist certain operators and syntax, perhaps including =, =>, dot notation to call member functions of an object, [ ] to access array elements, &. (b) Create a whitelist of allowable functions (e.g. math, and pluggable functions loaded, perhaps, using a method like that used in dTexts) (c) Create a list of allowable variable names (using either SGQA or question.title syntax) from questions that have already been (d) Add known token names to the list of allowable variables (e) Check that the equation only contains Strings, known variables, white-listed functions, and allowed punctuation. Otherwise throw an error, noting the offending part of the equation. (f) When replacing variables with their values, first check whether they are numeric or string. For Strings, surround them with quotes (so that users can't inject arbitrary code in an answer and have it evaluated). If the variable does not yet have a value, or is undefined, treat it as a String (e.g. [UNANSWERED: myVar]) to ease debugging (g) Validate that the string to be evaluated ends with ';' (h) Validate that parentheses match (to avoid one set of errors) (i) Consider using forking or other methods (like in http://php.net/manual/en/function.eval.php notes) so can catch errors at execution time. Otherwise, a FATAL error in the eval() will kill the rest of the script. I don't have a good sense of what sorts of syntax errors might be fatal (j) Use PHP eval() to evaluate the resulting string and return the resulting value. Are there other steps I need to consider to make this approach safe and secure? Alternatively, do you know of a well-supported, PHP-based equation parser or grammar generator I could use? (see notes above for what I've already explored)

TMSWhite 2011-05-27 20:21 reporter ~15138	Another option for (3) (since I'm having a hard time letting go of the idea of being able to do this in BNF). To recap, the functionality I'm looking for here is very basic: (1) All basic math operators and functions (2) Ability to call other functions from a "white list" of supported functions (which would be included from an external source - could be existing JavaScript/PHP functions, or external ones we need to add). At present, I categorize the functions by whether they take 0, 1, 2, 3, or unlimited arguments, so I can enforce a small degree of syntax checking. (3) Ability to handle numbers, strings, and dates separately (e.g. so can throw syntax exceptions if the a given math operator is not appropriate for a data type) (4) Only allowable syntax is supported (so can avoid calls to arrays, functions, hashes, macros, etc. - anything that might be unsafe or put the website at risk of an injection attack) (5) Only be able to reference known variables (e.g. question.title, SGQA, and token names). From a usability perspective, we want to (1) Validate that the equation is safe (2) Provide the user appropriate and helpful feedback if there is a syntax error. Once we have validated that the equation is safe, we could let PHP and/or JavaScript's eval() functions execute the code (after doing some token substitutions so that PHP/JavaScript lookup the answers properly based upon the question.title, SGQA, or token names). Also, I presume we only need to validate the equation using JavaScript, since the user will always use a web page to author the survey. Even if they import it from a file, we could always generate a test page that validates the equation syntax (using JavaScript) before final acceptance of the import. The only risk I see here is if a user manually edits the equation in the MySQL tables, but I presume that should be a minimal risk (e.g. you can trash a website you are managing, but not one belonging to someone else). So, are there good JavaScript-based expression evaluators / syntax checkers? I see that JSLint (http://www.jslint.com/) is written in JavaScript, available via GPL-equivalent on GitHub, and very well and recently documented. I'm going to see what other options are out there, and what it would take to trim down JSLint to validate the simplified equation syntax I'm targeting. Are there any concerns with this approach, should it pan out?

TMSWhite 2011-06-10 00:24 reporter ~15393	I have created a safe equation parser with complete unit tests. It is committed to the limesurvey_dev_tms branch under limesurvey/classes/eval. You can see the unit test results here: limesurvey/classes/eval/Test_ExpressionManager_Evaluate.php The system provides read-only access to registered variables (e.g. all pre-defined variables in the survey), and registered functions (e.g. all match functions, plus others we want to add, like sum(), if(test,if_true,if_false)) The parser can be changed to allow read-write access to variables if we want, or give the ability to create temporary variables, but both such options could cause unexpected side effects.

TMSWhite 2011-06-10 08:27 reporter ~15395	DONE, and fully unit tested. As per the commit message (revision 10245): Added embedded equation parser, which uses a recursive descent parser to support arbitrarily complex equations. The equation parser optionally provides detailed information about syntax errors, and safely returns NULL if user evaluates a poorly formed equation. The equation parser has read-only access to all previously asked survey variables via either question title or SGQA code. The equation parser currently provides safe access to ~100 PHP math and string functions, plus conditional logic via the if(test,true,false) function. The equation parser is called via the {EVAL: equation} syntax Also added {ANS: varName} syntax so can refer to variable by its question title instead of SGQA code. Fixed minor bug in dTexts.php that would cause {EVAL: equation } parsing to fail if there were a colon in the equation.

TMSWhite 2011-06-10 08:28 reporter	limesurvey_survey_showing_conditional_tailoring_using_equation_parser.lss (28,583 bytes)

TMSWhite 2011-06-10 08:31 reporter ~15396	The file limesurvey_survey_showing_conditional_tailoring_using_equation_parser.lss demonstrates how to use the new {EVAL: expression} and {ANS: varName} features. It asks 4 preliminary questions (name, age, number of kids, number of pets), then generates a tailored question based upon that information. Please give it a try.

TMSWhite 2011-06-10 09:06 reporter ~15398	Quick update to prior notes: In the end, I had to create my own compiler (a recursive descent parser) to safely evaluate equations. None of the compiler-compiler options would generate PHP code. Antlr looked most promising, but the plug-in for generating PHP was not well maintained. JSLint could not be easily modified to validate just the subset of functionality we needed, plus even if it did, it would still not address the PHP processing needs. So, I bit the bullet and ported my JavaCC-based equation parser into PHP, converting it into a recursive descent parser instead of creating a state based machine. Recursive descent parsers are much easier to read (it is only 1500 lines of code, a third of which are unit tests) than state based machines. The only down side is that equations can have a maximum of 10 levels of nested parentheses before PHP throws a nesting error (but how often does one need ten levels deep of nested parentheses)?

TMSWhite 2011-06-10 09:15 reporter	issue_05103.patch (64,510 bytes) # This patch file was generated by NetBeans IDE # Following Index: paths are relative to: C:\xampp\htdocs\limesurvey\classes # This patch can be applied using context Tools: Patch action on respective folder. # It uses platform neutral UTF-8 encoding and \n newlines. # Above lines and this line are ignored by the patching process. Index: dTexts/dFunctions/dFunctionAns.php --- dTexts/dFunctions/dFunctionAns.php Locally New +++ dTexts/dFunctions/dFunctionAns.php Locally New @@ -0,0 +1,29 @@ +<?php + +class dFunctionAns implements dFunctionInterface +{ + public function __construct() + { + } + + public function run($args) + { + global $connect; + $field = $args[0]; + if (isset($_SESSION['srid'])) $srid = $_SESSION['srid']; + $sid = returnglobal('sid'); + // Map Question.title to first SGQA field matching it + $fieldmap=createFieldMap($sid,$style='full'); + if (isset($fieldmap)) + { + foreach($fieldmap as $fielddata) + { + if ($fielddata['title'] == $field) + { + return retrieve_Answer($fielddata['fieldname'], $_SESSION['dateformats']['phpdate']); + } + } + } + return $field; + } +} Index: dTexts/dFunctions/dFunctionEval.php --- dTexts/dFunctions/dFunctionEval.php Locally New +++ dTexts/dFunctions/dFunctionEval.php Locally New @@ -0,0 +1,73 @@ +<?php + +include_once('classes/eval/ExpressionManager.php'); + +class dFunctionEval implements dFunctionInterface +{ + private $knownVars; + private $em; + + public function __construct() + { + } + + public function run($args) + { + global $connect; + $expr = htmlspecialchars_decode($args[0]); + if (isset($_SESSION['srid'])) $srid = $_SESSION['srid']; // what is this for? + $em = $this->getExpressionManager(); + $status = $em->Evaluate($expr); + $errs = $em->GetReadableErrors(); + $result = $em->GetResult(); + return $result; + } + + /** + * Return full list of variable names and values. + * TODO: Is there an existing function that does this? + * TODO: Want to only call this once per page refresh + * @return array + / + + private function getVarArray() + { + if (isset($this->knownVars)) { + return $this->knownVars; + } + + $sid = returnglobal('sid'); + $fieldmap=createFieldMap($sid,$style='full'); + $knownVars = array(); // mapping of VarName to Value + if (isset($fieldmap)) + { + foreach($fieldmap as $fielddata) + { + $value = retrieve_Answer($fielddata['fieldname'], $_SESSION['dateformats']['phpdate']); + $knownVars[$fielddata['title']] = $value; + $knownVars[$fielddata['fieldname']] = $value; + } + } + $this->knownVars = $knownVars; + return $this->knownVars; + } + + /* + * Goal is to create Expression Manager once per page refresh. + * @return <type> + / + + private function getExpressionManager() + { + if (isset($this->em)) + { + return $this->em; + } + + $em = new ExpressionManager(); + $varArray = $this->getVarArray(); + $em->RegisterVarnames($varArray); + $this->em = $em; + return $this->em; + } +} Index: dTexts/dTexts.php --- dTexts/dTexts.php Base (BASE) +++ dTexts/dTexts.php Locally Modified (Based On LOCAL) @@ -18,6 +18,7 @@ { $data=explode(':',$str); $funcName=array_shift($data); + $data = array(implode(':',$data)); try { $func = dTexts::loadFunction($funcName); @@ -39,7 +40,7 @@ / public static function loadFunction($name) { - $name=ucfirst(strtolower($name)); + $name=ucwords($name); $fileName='./classes/dTexts/dFunctions/dFunction'.$name.'.php'; if(!file_exists($fileName)) { Index: eval/ExpressionManager.php --- eval/ExpressionManager.php Locally New +++ eval/ExpressionManager.php Locally New @@ -0,0 +1,1505 @@ +<?php +/** + * Description of ExpressionManager + * Class which does safe evaluation of PHP expressions. Only registered functions and variables are allowed + * At present, all variables are read-only, but this could be extended to support creation of temporary variables and/or read-write access to registered variables + * + * @author Thomas M. White + / + +class ExpressionManager { + // These three variables are effectively static once constructed + private $asTokenType; + private $sTokenizerRegex; + private $asCategorizeTokensRegex; + private $amValidFunctions; // names and # params of valid functions + private $amVars; // names and optional values of valid variables + + // Thes variables are used while processing the equation + private $expr; // the source expression + private $tokens; // the list of generated tokens + private $count; // total number of $tokens + private $pos; // position within the $token array while processing equation + private $errs; // array of syntax errors + private $onlyparse; + private $stack; // stack of intermediate results + private $result; // final result of evaluating the expression; + private $evalStatus; // true if $result is a valid result, and there are no serious errors + private $varsUsed; // list of variables referenced in the equation + + function __construct() + { + // List of token-matching regular expressions + $regex_dq_string = '".?(?<!\\\\)"'; + $regex_sq_string = '\'.?(?<!\\\\)\''; + $regex_whitespace = '\s+'; + $regex_lparen = '$'; + $regex_rparen = '$'; + $regex_comma = ','; + $regex_unary = '\+\+\|--\|!'; + $regex_binary = '[+/-]'; + $regex_compare = '<=\|<\|>=\|>\|==\|!=\|\ble\b\|\blt\b\|\bge\b\|\bgt\b\|\beq\b\|\bne\b'; + $regex_assign = '=\|\+=\|-=\|\='; + $regex_word = '[A-Z][A-Z0-9_]'; + $regex_number = '[0-9]+\.?[0-9]\|\.[0-9]+'; + $regex_andor = '\band\b\|\bor\b\|&&\|\\|\\|'; + + // asTokenRegex and t_tokey_type must be kept in sync (same number and order) + $asTokenRegex = array( + $regex_dq_string, + $regex_sq_string, + $regex_whitespace, + $regex_lparen, + $regex_rparen, + $regex_comma, + $regex_andor, + $regex_compare, + $regex_word, + $regex_number, +// $regex_unary, +// $regex_assign, + $regex_binary, + ); + + $this->asTokenType = array( + 'STRING', + 'STRING', + 'SPACE', + 'LP', + 'RP', + 'COMMA', + 'AND_OR', + 'COMPARE', + 'WORD', + 'NUMBER', +// 'UNARYOP', +// 'ASSIGN', + 'BINARYOP', + ); + + // $sTokenizerRegex - a single regex used to split and equation into tokens + $this->sTokenizerRegex = '#(' . implode('\|',$asTokenRegex) . ')#i'; + + // $asCategorizeTokensRegex - an array of patterns so can categorize the type of token found - would be nice if could get this from preg_split + // Adding ability to capture 'OTHER' type, which indicates an error - unsupported syntax element + $this->asCategorizeTokensRegex = preg_replace("#^(.)$#","#^$1$#i",$asTokenRegex); + $this->asCategorizeTokensRegex[] = '/.+/'; + $this->asTokenType[] = 'OTHER'; + + // Each allowed function is a mapping from local name to external name + number of arguments + // Functions can have -1 (meaning unlimited), or a list of serveral allowable #s of arguments. + $this->amValidFunctions = array( + 'abs' =>array('abs','Absolute value',1), + 'acos' =>array('acos','Arc cosine',1), + 'acosh' =>array('acosh','Inverse hyperbolic cosine',1), + 'asin' =>array('asin','Arc sine',1), + 'asinh' =>array('asinh','Inverse hyperbolic sine',1), + 'atan2' =>array('atan2','Arc tangent of two variables',2), + 'atan' =>array('atan','Arc tangent',1), + 'atanh' =>array('atanh','Inverse hyperbolic tangent',1), + 'base_convert' =>array('base_convert','Convert a number between arbitrary bases',3), + 'bindec' =>array('bindec','Binary to decimal',1), + 'ceil' =>array('ceil','Round fractions up',1), + 'cos' =>array('cos','Cosine',1), + 'cosh' =>array('cosh','Hyperbolic cosine',1), + 'decbin' =>array('decbin','Decimal to binary',1), + 'dechex' =>array('dechex','Decimal to hexadecimal',1), + 'decoct' =>array('decoct','Decimal to octal',1), + 'deg2rad' =>array('deg2rad','Converts the number in degrees to the radian equivalent',1), + 'exp' =>array('exp','Calculates the exponent of e',1), + 'expm1' =>array('expm1','Returns exp(number) - 1, computed in a way that is accurate even when the value of number is close to zero',1), + 'floor' =>array('floor','Round fractions down',1), + 'fmod' =>array('fmod','Returns the floating point remainder (modulo) of the division of the arguments',2), + 'getrandmax' =>array('getrandmax','Show largest possible random value',0), + 'hexdec' =>array('hexdec','Hexadecimal to decimal',1), + 'hypot' =>array('hypot','Calculate the length of the hypotenuse of a right-angle triangle',2), + 'is_finite' =>array('is_finite','Finds whether a value is a legal finite number',1), + 'is_infinite' =>array('is_infinite','Finds whether a value is infinite',1), + 'is_nan' =>array('is_nan','Finds whether a value is not a number',1), + 'lcg_value' =>array('lcg_value','Combined linear congruential generator',0), + 'log10' =>array('log10','Base-10 logarithm',1), + 'log1p' =>array('log1p','Returns log(1 + number), computed in a way that is accurate even when the value of number is close to zero',1), + 'log' =>array('log','Natural logarithm',1,2), + 'max' =>array('max','Find highest value',-1), + 'min' =>array('min','Find lowest value',-1), + 'mt_getrandmax' =>array('mt_getrandmax','Show largest possible random value',0), + 'mt_rand' =>array('mt_rand','Generate a better random value',0,2), + 'mt_srand' =>array('mt_srand','Seed the better random number generator',0,1), + 'octdec' =>array('octdec','Octal to decimal',1), + 'pi' =>array('pi','Get value of pi',0), + 'pow' =>array('pow','Exponential expression',2), + 'rad2deg' =>array('rad2deg','Converts the radian number to the equivalent number in degrees',1), + 'rand' =>array('rand','Generate a random integer',0,2), + 'round' =>array('round','Rounds a float',1,2,3), + 'sin' =>array('sin','Sine',1), + 'sinh' =>array('sinh','Hyperbolic sine',1), + 'sqrt' =>array('sqrt','Square root',1), + 'srand' =>array('srand','Seed the random number generator',0,1), + 'sum' =>array('array_sum','Calculate the sum of values in an array',-1), + 'tan' =>array('tan','Tangent',1), + 'tanh' =>array('tanh','Hyperbolic tangent',1), + + 'empty' =>array('empty','Determine whether a variable is empty',1), + 'intval' =>array('intval','Get the integer value of a variable',1,2), + 'is_bool' =>array('is_bool','Finds out whether a variable is a boolean',1), + 'is_float' =>array('is_float','Finds whether the type of a variable is float',1), + 'is_int' =>array('is_int','Find whether the type of a variable is integer',1), + 'is_null' =>array('is_null','Finds whether a variable is NULL',1), + 'is_numeric' =>array('is_numeric','Finds whether a variable is a number or a numeric string',1), + 'is_scalar' =>array('is_scalar','Finds whether a variable is a scalar',1), + 'is_string' =>array('is_string','Find whether the type of a variable is string',1), + + 'addcslashes' =>array('addcslashes','Quote string with slashes in a C style',2), + 'addslashes' =>array('addslashes','Quote string with slashes',1), + 'bin2hex' =>array('bin2hex','Convert binary data into hexadecimal representation',1), + 'chr' =>array('chr','Return a specific character',1), + 'chunk_split' =>array('chunk_split','Split a string into smaller chunks',1,2,3), + 'convert_uudecode' =>array('convert_uudecode','Decode a uuencoded string',1), + 'convert_uuencode' =>array('convert_uuencode','Uuencode a string',1), + 'count_chars' =>array('count_chars','Return information about characters used in a string',1,2), + 'crc32' =>array('crc32','Calculates the crc32 polynomial of a string',1), + 'crypt' =>array('crypt','One-way string hashing',1,2), + 'hebrev' =>array('hebrev','Convert logical Hebrew text to visual text',1,2), + 'hebrevc' =>array('hebrevc','Convert logical Hebrew text to visual text with newline conversion',1,2), + 'html_entity_decode' =>array('html_entity_decode','Convert all HTML entities to their applicable characters',1,2,3), + 'htmlentities' =>array('htmlentities','Convert all applicable characters to HTML entities',1,2,3), + 'htmlspecialchars_decode' =>array('htmlspecialchars_decode','Convert special HTML entities back to characters',1,2), + 'htmlspecialchars' =>array('htmlspecialchars','Convert special characters to HTML entities',1,2,3,4), + 'implode' =>array('implode','Join array elements with a string',-1), + 'lcfirst' =>array('lcfirst','Make a string\'s first character lowercase',1), + 'levenshtein' =>array('levenshtein','Calculate Levenshtein distance between two strings',2,5), + 'ltrim' =>array('ltrim','Strip whitespace (or other characters) from the beginning of a string',1,2), + 'md5' =>array('md5','Calculate the md5 hash of a string',1), + 'metaphone' =>array('metaphone','Calculate the metaphone key of a string',1,2), + 'money_format' =>array('money_format','Formats a number as a currency string',1,2), + 'nl2br' =>array('nl2br','Inserts HTML line breaks before all newlines in a string',1,2), + 'number_format' =>array('number_format','Format a number with grouped thousands',1,2,4), + 'ord' =>array('ord','Return ASCII value of character',1), + 'quoted_printable_decode' =>array('quoted_printable_decode','Convert a quoted-printable string to an 8 bit string',1), + 'quoted_printable_encode' =>array('quoted_printable_encode','Convert a 8 bit string to a quoted-printable string',1), + 'quotemeta' =>array('quotemeta','Quote meta characters',1), + 'rtrim' =>array('rtrim','Strip whitespace (or other characters) from the end of a string',1,2), + 'sha1' =>array('sha1','Calculate the sha1 hash of a string',1), + 'similar_text' =>array('similar_text','Calculate the similarity between two strings',1,2), + 'soundex' =>array('soundex','Calculate the soundex key of a string',1), + 'sprintf' =>array('sprintf','Return a formatted string',-1), + 'str_ireplace' =>array('str_ireplace','Case-insensitive version of str_replace',3), + 'str_pad' =>array('str_pad','Pad a string to a certain length with another string',2,3,4), + 'str_repeat' =>array('str_repeat','Repeat a string',2), + 'str_replace' =>array('str_replace','Replace all occurrences of the search string with the replacement string',3), + 'str_rot13' =>array('str_rot13','Perform the rot13 transform on a string',1), + 'str_shuffle' =>array('str_shuffle','Randomly shuffles a string',1), + 'str_word_count' =>array('str_word_count','Return information about words used in a string',1), + 'strcasecmp' =>array('strcasecmp','Binary safe case-insensitive string comparison',2), + 'strcmp' =>array('strcmp','Binary safe string comparison',2), + 'strcoll' =>array('strcoll','Locale based string comparison',2), + 'strcspn' =>array('strcspn','Find length of initial segment not matching mask',2,3,4), + 'strip_tags' =>array('strip_tags','Strip HTML and PHP tags from a string',1,2), + 'stripcslashes' =>array('stripcslashes','Un-quote string quoted with addcslashes',1), + 'stripos' =>array('stripos','Find position of first occurrence of a case-insensitive string',2,3), + 'stripslashes' =>array('stripslashes','Un-quotes a quoted string',1), + 'stristr' =>array('stristr','Case-insensitive strstr',2,3), + 'strlen' =>array('strlen','Get string length',1), + 'strnatcasecmp' =>array('strnatcasecmp','Case insensitive string comparisons using a "natural order" algorithm',2), + 'strnatcmp' =>array('strnatcmp','String comparisons using a "natural order" algorithm',2), + 'strncasecmp' =>array('strncasecmp','Binary safe case-insensitive string comparison of the first n characters',3), + 'strncmp' =>array('strncmp','Binary safe string comparison of the first n characters',3), + 'strpbrk' =>array('strpbrk','Search a string for any of a set of characters',2), + 'strpos' =>array('strpos','Find position of first occurrence of a string',2,3), + 'strrchr' =>array('strrchr','Find the last occurrence of a character in a string',2), + 'strrev' =>array('strrev','Reverse a string',1), + 'strripos' =>array('strripos','Find position of last occurrence of a case-insensitive string in a string',2,3), + 'strrpos' =>array('strrpos','Find the position of the last occurrence of a substring in a string',2,3), + 'strspn' =>array('Finds the length of the initial segment of a string consisting entirely of characters contained within a given mask.',2,3,4), + 'strstr' =>array('strstr','Find first occurrence of a string',2,3), + 'strtolower' =>array('strtolower','Make a string lowercase',1), + 'strtoupper' =>array('strtoupper','Make a string uppercase',1), + 'strtr' =>array('strtr','Translate characters or replace substrings',3), + 'substr_compare'=>array('substr_compare','Binary safe comparison of two strings from an offset, up to length characters',3,4,5), + 'substr_count' =>array('substr_count','Count the number of substring occurrences',2,3,4), + 'substr_replace'=>array('substr_replace','Replace text within a portion of a string',3,4), + 'substr' =>array('substr','Return part of a string',2,3), + 'ucfirst' =>array('ucfirst','Make a string\'s first character uppercase',1), + 'ucwords' =>array('ucwords','Uppercase the first character of each word in a string',1), + + 'stddev' =>array('stats_standard_deviation','Returns the standard deviation',-1), + + // Locally declared functions + 'if' => array('exprmgr_if','Excel-style if(test,result_if_true,result_if_false)',3), + 'list' => array('exprmgr_list','Return comma-separated list of values',-1), + ); + + $this->amVars = array(); + + } + + /** + * Add an error to the error log + * + * @param <type> $errMsg + * @param <type> $token + / + private function AddError($errMsg, $token) + { + $this->errs[] = array($errMsg, $token); + } + + /* + * EvalBinary() computes binary expressions, such as (a or b), (c * d), popping the top two entries off the + * stack and pushing the result back onto the stack. + * + * @param array $token + * @return boolean - false if there is any error, else true + / + + private function EvalBinary(array $token) + { + if (count($this->stack) < 2) + { + $this->AddError("Unable to evaluate binary operator - fewer than 2 entries on stack", $token); + return false; + } + $arg2 = $this->StackPop(); + $arg1 = $this->StackPop(); + if (is_null($arg1) or is_null($arg2)) + { + $this->AddError("Invalid value(s) on the stack", $token); + return false; + } + // TODO: try to determine datatype? + switch(strtolower($token[0])) + { + case 'or': + case '\|\|': + $result = array(($arg1[0] or $arg2[0]),$token[1],'NUMBER'); + break; + case 'and': + case '&&': + $result = array(($arg1[0] and $arg2[0]),$token[1],'NUMBER'); + break; + case '==': + case 'eq': + $result = array(($arg1[0] == $arg2[0]),$token[1],'NUMBER'); + break; + case '!=': + case 'ne': + $result = array(($arg1[0] != $arg2[0]),$token[1],'NUMBER'); + break; + case '<': + case 'lt': + $result = array(($arg1[0] < $arg2[0]),$token[1],'NUMBER'); + break; + case '<='; + case 'le': + $result = array(($arg1[0] <= $arg2[0]),$token[1],'NUMBER'); + break; + case '>': + case 'gt': + $result = array(($arg1[0] > $arg2[0]),$token[1],'NUMBER'); + break; + case '>='; + case 'ge': + $result = array(($arg1[0] >= $arg2[0]),$token[1],'NUMBER'); + break; + case '+': + $result = array(($arg1[0] + $arg2[0]),$token[1],'NUMBER'); + break; + case '-': + $result = array(($arg1[0] - $arg2[0]),$token[1],'NUMBER'); + break; + case '': + $result = array(($arg1[0] * $arg2[0]),$token[1],'NUMBER'); + break; + case '/'; + $result = array(($arg1[0] / $arg2[0]),$token[1],'NUMBER'); + break; + } + $this->StackPush($result); + return true; + } + + /** + * Processes operations like +a, -b, !c + * @param array $token + * @return boolean - true if success, false if any error occurred + / + + private function EvalUnary(array $token) + { + if (count($this->stack) < 1) + { + $this->AddError("Unable to evaluate unary operator - no entries on stack", $token); + return false; + } + $arg1 = $this->StackPop(); + if (is_null($arg1)) + { + $this->AddError("Invalid value(s) on the stack", $token); + return false; + } + // TODO: try to determine datatype? + switch($token[0]) + { + case '+': + $result = array((+$arg1[0]),$token[1],'NUMBER'); + break; + case '-': + $result = array((-$arg1[0]),$token[1],'NUMBER'); + break; + case '!'; + $result = array((!$arg[0]),$token[1],'NUMBER'); + break; + } + $this->StackPush($result); + return true; + } + + + /* + * Main entry function + * @param <type> $expr + * @param <type> $onlyparse - if true, then validate the syntax without computing an answer + * @return boolean - true if success, false if any error occurred + / + + public function Evaluate($expr, $onlyparse=false) + { + $this->expr = $expr; + $this->tokens = $this->amTokenize($expr); + $this->count = count($this->tokens); + $this->pos = -1; // starting position within array (first act will be to increment it) + $this->errs = array(); + $this->onlyparse = $onlyparse; + $this->stack = array(); + $this->evalStatus = false; + $this->result = NULL; + $this->varsUsed = array(); + + if ($this->HasSyntaxErrors()) { + return false; + } + else if ($this->EvaluateExpressions()) + { + if ($this->pos < $this->count) + { + $this->AddError("Extra tokens found starting at", $this->tokens[$this->pos]); + return false; + } + $this->result = $this->StackPop(); + if (is_null($this->result)) + { + return false; + } + if (count($this->stack) == 0) + { + $this->evalStatus = true; + return true; + } + else + { + $this-AddError("Unbalanced equation - values left on stack",NULL); + return false; + } + } + else + { + $this->AddError("Not a valid expression",NULL); + return false; + } + } + + + /* + * Process "a op b" where op in (+,-,concatenate) + * @return boolean - true if success, false if any error occurred + / + private function EvaluateAdditiveExpression() + { + if (!$this->EvaluateMultiplicativeExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '+': + case '-'; + if ($this->EvaluateMultiplicativeExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue; + } + else + { + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process a Constant (number of string), retrieve the value of a known variable, or process a function, returning result on the stack. + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateConstantVarOrFunction() + { + if ($this->pos + 1 >= $this->count) + { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + switch ($token[2]) + { + case 'NUMBER': + case 'STRING': + $this->StackPush($token); + return true; + break; + case 'WORD': + if (($this->pos + 1) < $this->count and $this->tokens[($this->pos + 1)][2] == 'LP') + { + return $this->EvaluateFunction(); + } + else + { + if ($this->isValidVariable($token[0])) + { + $this->varsUsed[] = $token[0]; // add this variable to list of those used in this equation + $result = array($this->amVars[$token[0]],$token[1],'NUMBER'); + $this->StackPush($result); + return true; + } + else + { + $this->AddError("Undefined Variable", $token); + return false; + } + } + break; + case 'COMMA': + --$this->pos; + $this->AddError("Should never get to this line?",$token); + return false; + default: + return false; + break; + } + } + + /* + * Process "a == b", "a eq b", "a != b", "a ne b" + * @return boolean - true if success, false if any error occurred + / + private function EvaluateEqualityExpression() + { + if (!$this->EvaluateRelationExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '==': + case 'eq': + case '!=': + case 'ne': + if ($this->EvaluateRelationExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue; + } + else + { + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process a single expression (e.g. without commas) + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateExpression() + { + if ($this->pos + 2 < $this->count) + { + $token1 = $this->tokens[++$this->pos]; + $token2 = $this->tokens[++$this->pos]; + if ($this->isValidVariable($token1[0]) and $token2[0] == '=') + { + $evalStatus = $this->EvaluateLogicalOrExpression(); + if ($evalStatus) + { + $result = $this->StackPop(); + if (!is_null($result)) + { + $this->setVariableValue($token1[0], $result[1]); + $this->StackPush($result); // push result back on stack + } + else + { + $evalStatus = false; + } + } + return $evalStatus; + } + else + { + // not an assignment expression, so try something else + $this->pos -= 2; + return $this->EvaluateLogicalOrExpression(); + } + } + else + { + return $this->EvaluateLogicalOrExpression(); + } + } + + /* + * Process "expression [, expression]* + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateExpressions() + { + $evalStatus = $this->EvaluateExpression(); + if (!$evalStatus) + { + return false; + } + + while (++$this->pos < $this->count) { // TODO - would this be clearer as ($this->pos + 1)? + $token = $this->tokens[$this->pos]; + if ($token[2] == 'RP') + { + return true; // presumbably the end of an expression + } + else if ($token[2] == 'COMMA') + { + if ($this->EvaluateExpression()) + { + $secondResult = $this->StackPop(); + $firstResult = $this->StackPop(); + if (is_null($firstResult)) + { + return false; + } + $this->StackPush($secondResult); + $evalStatus = true; + } + + } + else + { + $this->AddError("Expected expressions separated by commas",$token); + $evalStatus = false; + break; + } + } + while (++$this->pos < $this->count) + { + $token = $this->tokens[$this->pos]; // TODO - would this be clearer as ($this->pos + 1)? + $this->AddError("Extra token found after Expressions",$token); + $evalStatus = false; + } + return $evalStatus; + } + + /* + * Process a function call + * @return boolean - true if success, false if any error occurred + / + private function EvaluateFunction() + { + $funcNameToken = $this->tokens[$this->pos]; // note that don't need to increment position for functions + $funcName = $funcNameToken[0]; + if (!$this->isValidFunction($funcName)) + { + $this->AddError("Undefined Function", $funcNameToken); + return false; + } + $token2 = $this->tokens[++$this->pos]; + if ($token2[2] != 'LP') + { + $this->AddError("Expected '(' after function name", $token); + } + $params = array(); // will just store array of values, not tokens + while ($this->pos + 1 < $this->count) + { + $token3 = $this->tokens[$this->pos + 1]; + if (count($params) > 0) + { + // should have COMMA or RP + if ($token3[2] == 'COMMA') + { + ++$this->pos; // consume the token so can process next clause + if ($this->EvaluateExpression()) + { + $value = $this->StackPop(); + if (is_null($value)) + { + return false; + } + $params[] = $value[0]; + continue; + } + else + { + $this->AddError("Extra comma found in function", $token3); + return false; + } + } + } + if ($token3[2] == 'RP') + { + ++$this->pos; // consume the token so can process next clause + return $this->RunFunction($funcNameToken,$params); + } + else + { + if ($this->EvaluateExpression()) + { + $value = $this->StackPop(); + if (is_null($value)) + { + return false; + } + $params[] = $value[0]; + continue; + } + else + { + // TODO - what type of syntax error goes here? Or assume proper message will be thrown? + return false; + } + } + } + } + + /* + * Process "a && b" or "a and b" + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateLogicalAndExpression() + { + if (!$this->EvaluateEqualityExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '&&': + case 'and': + if ($this->EvaluateEqualityExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process "a \|\| b" or "a or b" + * @return boolean - true if success, false if any error occurred + / + private function EvaluateLogicalOrExpression() + { + if (!$this->EvaluateLogicalAndExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '\|\|': + case 'or': + if ($this->EvaluateLogicalAndExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + default: + // no more expressions being ORed together, so continue parsing + --$this->pos; + return true; + } + } + // no more tokens to parse + return true; + } + + /* + * Process "a op b" where op in (,/) + @return boolean - true if success, false if any error occurred + / + + private function EvaluateMultiplicativeExpression() + { + if (!$this->EvaluateUnaryExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '': + case '/'; + if ($this->EvaluateUnaryExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /** + * Process expressions including functions and parenthesized blocks + * @return boolean - true if success, false if any error occurred + / + + private function EvaluatePrimaryExpression() + { + if (($this->pos + 1) >= $this->count) { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + if ($token[2] == 'LP') + { + if (!$this->EvaluateExpressions()) + { + return false; + } + / + if ($this->pos+1 >= $this->count) + { + $token = $this->tokens[$this->pos]; + $this->AddError("Expected ')'",$token); + return false; + } + / + $token = $this->tokens[$this->pos]; // TODO - would this be clearer as ++$this->pos + if ($token[2] == 'RP') + { + return true; + } + else + { + $this->AddError("Expected ')'", $token); + return false; + } + } + else + { + --$this->pos; + return $this->EvaluateConstantVarOrFunction(); + } + } + + /* + * Process "a op b" where op in (lt, gt, le, ge, <, >, <=, >=) + * @return boolean - true if success, false if any error occurred + / + private function EvaluateRelationExpression() + { + if (!$this->EvaluateAdditiveExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '<': + case 'lt': + case '<='; + case 'le': + case '>': + case 'gt': + case '>='; + case 'ge': + if ($this->EvaluateAdditiveExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process "op a" where op in (+,-,!) + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateUnaryExpression() + { + if (($this->pos + 1) >= $this->count) { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '+': + case '-': + case '!': + if (!$this->EvaluatePrimaryExpression()) + { + return false; + } + return $this->EvalUnary($token); + break; + default: + --$this->pos; + return $this->EvaluatePrimaryExpression(); + } + } + + /* + * Return the result of evaluating the equation - NULL if error + * @return mixed + / + public function GetResult() + { + return $this->result[0]; + } + + /* + * Return an array of errors + * @return array + / + public function GetErrors() + { + return $this->errs; + } + + /* + * Return an array of human-readable errors (message, offending token, offset of offending token within equation) + * @return array + / + public function GetReadableErrors() + { + $errs = array(); + foreach ($this->errs as $err) + { + $msg = $err[0]; + $token = $err[1]; + $toshow = 'ERR'; + if (!is_null($token)) + { + $toshow .= '[' . $token[0] . ' @pos=' . $token[1] . ']'; + } + $toshow .= ': ' . $msg; + $errs[] = $toshow; + } + return $errs; + } + + /* + * Return array of the list of variables used in the equation + * @return array + / + public function GetVarsUsed() + { + return array_unique($this->varsUsed); + } + + /* + * Return true if there were syntax or processing errors + * @return boolean + / + public function HasErrors() + { + return (count($this->errs) > 0); + } + + /* + * Return true if there are syntax errors + * @return boolean + / + private function HasSyntaxErrors() + { + // check for bad tokens + // check for unmatched parentheses + // check for undefined variables + // check for undefined functions (but can't easily check allowable # elements?) + + $nesting = 0; + + for ($i=0;$i<$this->count;++$i) + { + $token = $this->tokens[$i]; + switch ($token[2]) + { + case 'LP': + ++$nesting; + break; + case 'RP': + --$nesting; + if ($nesting < 0) + { + $this->AddError("Extra ')' detected", $token); + } + break; + case 'WORD': + if ($i+1 < $this->count and $this->tokens[$i+1][2] == 'LP') + { + if (!$this->isValidFunction($token[0])) + { + $this->AddError("Undefined function", $token); + } + } + else + { + if (!$this->isValidVariable($token[0])) + { + $this->AddError("Undefined variable", $token); + } + } + break; + case 'OTHER': + $this->AddError("Unsupported syntax", $token); + break; + default: + break; + } + } + if ($nesting != 0) + { + $this->AddError("Parentheses not balanced",NULL); + } + return (count($this->errs) > 0); + } + + /* + * Return true if the function name is registered + * @param <type> $name + * @return boolean + / + + private function isValidFunction($name) + { + return array_key_exists($name,$this->amValidFunctions); + } + + /* + * Return true if the variable name is registered + * @param <type> $name + * @return boolean + / + private function isValidVariable($name) + { + return array_key_exists($name,$this->amVars); + } + + /* + * Set the value of a registered variable + * @param <type> $name + * @param <type> $value + / + private function setVariableValue($name,$value) + { + // TODO - set this externally + $this->amVars[$name] = $value; + } + + /* + * Run a registered function + * @param <type> $funcNameToken + * @param <type> $params + * @return boolean + / + private function RunFunction($funcNameToken,$params) + { + $name = $funcNameToken[0]; + if (!$this->isValidFunction($name)) + { + return false; + } + $func = $this->amValidFunctions[$name]; + $funcName = $func[0]; + $numArgs = count($params); + + if (function_exists($funcName)) { + $numArgsAllowed = array_slice($func, 2); + $argsPassed = is_array($params) ? count($params) : 0; + + // for unlimited # parameters + try + { + if (in_array(-1, $numArgsAllowed)) { + $result = $funcName($params); + + // Call function with the params passed + } elseif (in_array($argsPassed, $numArgsAllowed)) { + + switch ($argsPassed) { + case 0: + $result = $funcName(); + break; + case 1: + $result = $funcName($params[0]); + break; + case 2: + $result = $funcName($params[0], $params[1]); + break; + case 3: + $result = $funcName($params[0], $params[1], $params[2]); + break; + case 4: + $result = $funcName($params[0], $params[1], $params[2], $params[3]); + break; + default: + $this->AddError("Error: Unsupported arg count: $funcName(".implode(", ",$params),$funcNameToken); + return false; + } + + } else { + $this->AddError("Error: Incorrect arg count: " . $funcName ."(".implode(", ",$params).")",$funcNameToken); + return false; + } + } + catch (Exception $e) + { + $this->AddError($e->getMessage(),$funcNameToken); + return false; + } + $token = array($result,$funcNameToken[1],'NUMBER'); + $this->StackPush($token); + return true; + } + } + + /* + * Add user functions to array of allowable functions within the equation. + * $functions is an array of key to value mappings like this: + * 'newfunc' => array('my_func_script', 1,3) + * where 'newfunc' is the name of an allowable function wihtin the expression, 'my_func_script' is the registered PHP function name, + * and 1,3 are the list of allowable numbers of paremeters (so my_func() can take 1 or 3 parameters. + * + * @param array $functions + / + + public function RegisterFunctions(array $functions) { + $this->amValidFunctions= array_merge($this->amValidFunctions, $functions); + } + + /* + * Add list of allowable variable names within the equation + * $varnames is an array of key to value mappings like this: + * 'myvar' => value + * where value is optional (e.g. can be blank), and can be any scalar type (e.g. string, number, but not array) + * the system will use the values as fast lookup when doing calculations, but if it needs to set values, it will call + * the interface function to set the values by name + * + * @param array $varnames + / + public function RegisterVarnames(array $varnames) { + $this->amVars = array_merge($this->amVars, $varnames); + } + + /* + * Pop a value token off of the stack + * @return token + / + + private function StackPop() + { + if (count($this->stack) > 0) + { + return array_pop($this->stack); + } + else + { + $this->AddError("Tried to pop value off of empty stack", NULL); + return NULL; + } + } + + /* + * Stack only holds values (number, string), not operators + * @param array $token + / + + private function StackPush(array $token) + { + if ($this->onlyparse) + { + // If only parsing, still want to validate syntax, so use "1" for all variables + switch($token[2]) + { + case 'STRING': + $this->stack[] = array(1,$token[1],'STRING'); + break; + case 'NUMBER': + default: + $this->stack[] = array(1,$token[1],'NUMBER'); + break; + } + } + else + { + $this->stack[] = $token; + } + } + + /* + * Split the source string into tokens, removing whitespace, and categorizing them by type. + * + * @param $src + * @return array + / + + private function amTokenize($src) + { + // $tokens0 = array of tokens from equation, showing value and offset position. Will include SPACE, which should be removed + $tokens0 = preg_split($this->sTokenizerRegex,$src,-1,(PREG_SPLIT_NO_EMPTY \| PREG_SPLIT_DELIM_CAPTURE \| PREG_SPLIT_OFFSET_CAPTURE)); + + // $tokens = array of tokens from equation, showing value, offsete position, and type. Will not contain SPACE, but will contain OTHER + $tokens = array(); + // Add token_type to $tokens: For each token, test each categorization in order - first match will be the best. + for ($j=0;$j<count($tokens0);++$j) + { + for ($i=0;$i<count($this->asCategorizeTokensRegex);++$i) + { + $token = $tokens0[$j][0]; + if (preg_match($this->asCategorizeTokensRegex[$i],$token)) + { + if ($this->asTokenType[$i] !== 'SPACE') { + $tokens0[$j][2] = $this->asTokenType[$i]; + if ($this->asTokenType[$i] == 'STRING') + { + // remove outside quotes + $unquotedToken = stripslashes(substr($token,1,-1)); + $tokens0[$j][0] = $unquotedToken; + } + $tokens[] = $tokens0[$j]; // get first matching non-SPACE token type and push onto $tokens array + } + break; // only get first matching token type + } + } + } + return $tokens; + } + + /* + * Unit test the Tokenizer - Tokenize and generate a HTML-compatible print-out of a comprehensive set of test cases + / + + static function UnitTestTokenizer() + { + // Comprehensive test cases for tokenizing + $tests = <<<EOD + String: "Can strings contain embedded \"quoted passages\" (and parenthesis + other characters?)?" + String: "can single quoted strings" . 'contain nested \'quoted sections\'?'; + Parens: upcase('hello'); + Numbers: 42 72.35 -15 +37 42A .5 0.7 + And_Or: (this and that or the other); Sandles, sorting; (a && b \|\| c) + Words: hi there, my name is C3PO! + UnaryOps: ++a, --b + BinaryOps: (a + b c / d) + Comparators: > >= < <= == != gt ge lt le eq ne (target large gents built agile less equal) + Assign: = += -= = + Errors: Apt # 10C; (2 > 0) ? 'hi' : 'there'; array[30]; >>> <<< / this is not a comment / // neither is this +EOD; + + $em = new ExpressionManager(); + + foreach(explode("\n",$tests) as $test) + { + $tokens = $em->amTokenize($test); + print '<b>' . $test . '</b><hr/>'; + print '<code>'; + print implode("<br/>\n",explode("\n",print_r($tokens,TRUE))); + print '</code><hr/>'; + } + } + + /* + * Unit test the Evaluator, allowing for passing in of extra functions, variables, and tests + * @param array $extraFunctions + * @param array $extraVars + * @param <type> $extraTests + / + + static function UnitTestEvaluator(array $extraFunctions=array(), array $extraVars=array(), $extraTests='1:1') + { + // Some test cases for Evaluator + $vars = array( + 'one' =>1, + 'two' =>2, + 'three' =>3, + 'four' =>4, + 'five' =>5, + 'six' =>6, + 'seven' =>7, + 'eight' =>8, + 'nine' =>9, + 'ten' =>10, + 'eleven' => 11, + 'twelve' => 12, + 'half' =>.5, + 'hi' =>'there', + 'hello' =>"Tom", + 'a' =>0, + 'b' =>0, + 'c' =>0, + 'd' =>0, + ); + + // Syntax for $tests is: + // expectedResult:expression + // if the expected result is an error, use NULL for the expected result + $tests = <<<EOD +2:max(one,two) +5:max(one,two,three,four,five) +1024:max(one,(twothree),pow(four,five),six) +1:min(one,two,three,four,five) +27:pow(3,3) +5:hypot(three,four) +0:0 +24:one * two * three * four +-4:five - four - three - two +0:two * three - two - two - two +4:two * three - two +3.1415926535898:pi() +1:pi() == pi() * 2 - pi() +1:sin(pi()/2) +1:sin(0.5 * pi()) +1:sin(pi()/2) == sin(.5 * pi()) +105:5 + 1, 7 * 15 +7:7 +15:10 + 5 +24:12 * 2 +10:13 - 3 +3.5:14 / 4 +5:3 + 1 * 2 +1:one +there:hi +6.25:one * two - three / four + five +1:one + hi +1:two > one +1:two gt one +1:three >= two +1:three ge two +0:four < three +0:four lt three +0:four <= three +0:four le three +0:four == three +0:four eq three +1:four != three +0:four ne four +0:one * hi +5:abs(-five) +0:acos(pi()/2) +0:asin(pi()/2) +10:ceil(9.1) +9:floor(9.9) +32767:getrandmax() +0:rand() +15:sum(one,two,three,four,five) +5:intval(5.7) +1:is_float('5.5') +0:is_float('5') +1:is_numeric(five) +0:is_numeric(hi) +1:is_string(hi) +2.4:(one * two) + (three * four) / (five * six) +1:(one * (two + (three - four) + five) / six) +0:one && 0 +0:two and 0 +1:five && 6 +1:seven && eight +1:one or 0 +1:one \|\| 0 +1:(one and 0) \|\| (two and three) +NULL:hi(there); +NULL:(one * two + (three - four) +NULL:(one * two + (three - four))) +NULL:a=five +NULL:++a +NULL:--b +NULL:c=a +NULL:c=five +NULL:c/=three +NULL:c-=six +11:eleven +144:twelve twelve +4:if(5 > 7,2,4) +there:if((one > two),'hi','there') +64:if((one < two),pow(2,6),pow(6,2)) +1,2,3,4,5:list(one,two,three,min(four,five,six),max(three,four,five)) +11,12:list(eleven,twelve) +EOD; + + $em = new ExpressionManager(); + $em->RegisterVarnames($vars); + + if (is_array($extraVars) and count($extraVars) > 0) + { + $em->RegisterVarnames($extraVars); + } + if (is_array($extraFunctions) and count($extraFunctions) > 0) + { + $em->RegisterFunctions($extraFunctions); + } + if (is_string($extraTests)) + { + $tests .= "\n" . $extraTests; + } + + print '<table border="1"><tr><th>Expression</th><th>Result</th><th>Expected</th><th>VarsUsed</th><th>Errors</th></tr>'; + foreach(explode("\n",$tests)as $test) + { + $values = explode(":",$test); + $expectedResult = array_shift($values); + $expr = implode(":",$values); + $resultStatus = 'ok'; + print '<tr><td>' . $expr . "</td>\n"; + $status = $em->Evaluate($expr); + $result = $em->GetResult(); + $valToShow = $result; + if (is_null($result)) { + $valToShow = "NULL"; + } + print '<td>' . $valToShow . "</td>\n"; + if ($valToShow != $expectedResult) + { + $resultStatus = 'error'; + } + print "<td class='" . $resultStatus . "'>" . $expectedResult . "</td>\n"; + $varsUsed = $em->GetVarsUsed(); + if (is_array($varsUsed) and count($varsUsed) > 0) { + print '<td>' . implode(', ', $varsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $errs = $em->GetReadableErrors(); + $errStatus = 'ok'; + if (is_array($errs) and count($errs) > 0) { + if ($expectedResult != "NULL") + { + $errStatus = 'error'; // should have been error free + } + print "<td class='" . $errStatus . "'>" . implode("<br/>\n", $errs) . "</td>\n"; + } + else { + if ($expectedResult == "NULL") + { + $errStatus = 'error'; // should have had errors + } + print "<td class='" . $errStatus . "'> </td>\n"; + } + print '</tr>'; + } + print '</table>'; + } +} + +/* + * Extra Functions can go here. TODO: Find good way to inlcude these extra functions externally. + * Tried via ExpressionManagerFunctions, but they weren't properly included in dFunctionEval.php + / + +function exprmgr_if($test,$ok,$error) +{ + if ($test) + { + return $ok; + } + else + { + return $error; + } +} + +function exprmgr_list($args) +{ + return implode(",",$args); +} + + +?> Index: eval/ExpressionManagerFunctions.php --- eval/ExpressionManagerFunctions.php Locally New +++ eval/ExpressionManagerFunctions.php Locally New @@ -0,0 +1,51 @@ +<?php +/* + * Put functions that you want visibile within Expression Manager in this file + * + * @author Thomas M. White + / + +// Each allowed function is a mapping from local name to external name + number of arguments +// Functions can have -1 (meaning unlimited), or a list of serveral allowable #s of arguments. +$exprmgr_functions = array( + 'if' => array('exprmgr_if','Excel-style if(test,result_if_true,result_if_false)',3), + 'list' => array('exprmgr_list','Return comma-separated list of values',-1), +); + +// Extra static variables for unit tests +$exprmgr_extraVars = array( + 'eleven' => 11, + 'twelve' => 12, +); + +// Unit tests of any added functions +$exprmgr_extraTests = <<<EOD +11:eleven +144:twelve twelve +4:if(5 > 7,2,4) +'there':if((one > two),'hi','there') +64:if((one < two),pow(2,6),pow(6,2)) +1,2,3,4,5:list(one,two,three,min(four,five,six),max(three,four,five)) +11,12:list(eleven,twelve) +EOD; + +function exprmgr_if($test,$ok,$error) +{ + if ($test) + { + return $ok; + } + else + { + return $error; + } +} + +function exprmgr_list($args) +{ + return implode(",",$args); +} + + + +?> Index: eval/Test_ExpressionManager_Evaluate.php --- eval/Test_ExpressionManager_Evaluate.php Locally New +++ eval/Test_ExpressionManager_Evaluate.php Locally New @@ -0,0 +1,29 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <style type="text/css"> + <!-- +.error { + background-color: #ff0000; +} +.ok { + background-color: #00ff00 +} + --> + </style> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; +// include 'ExpressionManagerFunctions.php'; +// ExpressionManager::UnitTestEvaluator($exprmgr_functions,$exprmgr_extraVars,$exprmgr_extraTests); + ExpressionManager::UnitTestEvaluator(); + ?> + </body> +</html> Index: eval/Test_ExpressionManager_Tokenizer.php --- eval/Test_ExpressionManager_Tokenizer.php Locally New +++ eval/Test_ExpressionManager_Tokenizer.php Locally New @@ -0,0 +1,17 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; + ExpressionManager::UnitTestTokenizer(); + ?> + </body> +</html> issue_05103.patch (64,510 bytes)

TMSWhite 2011-06-10 09:20 reporter ~15399	Please test the attached unified patch (issue_05103.patch). It was generated against limesurvey (main branch) revision 10245. I have tested the limesurvey_survey_showing_conditional_tailoring_using_equation_parser.lss survey and unit tests (http://localhost/limesurvey/classes/eval/Test_ExpressionManager_Evaluate.php) after applying this patch, and they all run fine for me. So, if this also works for you, please add this feature to the main branch and mark this issue 'resolved'. Thanks.

TMSWhite 2011-06-11 21:49 reporter ~15411	What is the preferred way to set values of variables in the database? Several of the other threads seem they could benefit from this,so I have upgraded the equation parser to support assigning registered variables within an equation (e.g. a = b, c += d, etc). The unit testing on the equation parser works fine, but all assignment is done locally within the equation parser. To integrate with LimeSurvey, I'll set a dirty flag so that we know which of the registered variables had had their values changed. So, once I know which of the variables have changed values, how do I ensure that this gets recorded? (1) Server-Side (PHP) - Do I have to update the database directly? (2) Client-Side (JavaScript) - If I just update the value of the appropriate DOM object (e.g. document.getElementById("answerSGQA").value = new_value, where SGQA is the SGQA code), will LimeSurvey automatically update the associated database field, or do I need to do something else too?

TMSWhite 2011-06-12 09:38 reporter	issue_05103-rev2.patch (83,074 bytes) # This patch file was generated by NetBeans IDE # Following Index: paths are relative to: C:\xampp\htdocs\limesurvey\classes # This patch can be applied using context Tools: Patch action on respective folder. # It uses platform neutral UTF-8 encoding and \n newlines. # Above lines and this line are ignored by the patching process. Index: dTexts/dFunctions/dFunctionAns.php --- dTexts/dFunctions/dFunctionAns.php Locally New +++ dTexts/dFunctions/dFunctionAns.php Locally New @@ -0,0 +1,29 @@ +<?php + +class dFunctionAns implements dFunctionInterface +{ + public function __construct() + { + } + + public function run($args) + { + global $connect; + $field = $args[0]; + if (isset($_SESSION['srid'])) $srid = $_SESSION['srid']; + $sid = returnglobal('sid'); + // Map Question.title to first SGQA field matching it + $fieldmap=createFieldMap($sid,$style='full'); + if (isset($fieldmap)) + { + foreach($fieldmap as $fielddata) + { + if ($fielddata['title'] == $field) + { + return retrieve_Answer($fielddata['fieldname'], $_SESSION['dateformats']['phpdate']); + } + } + } + return $field; + } +} Index: dTexts/dFunctions/dFunctionEval.php --- dTexts/dFunctions/dFunctionEval.php Locally New +++ dTexts/dFunctions/dFunctionEval.php Locally New @@ -0,0 +1,91 @@ +<?php + +include_once('classes/eval/ExpressionManager.php'); + +class dFunctionEval implements dFunctionInterface +{ + private $knownVars; + private $knownReservedWords; + private $em; + + public function __construct() + { + } + + public function run($args) + { + global $connect; + $expr = htmlspecialchars_decode($args[0]); + if (isset($_SESSION['srid'])) $srid = $_SESSION['srid']; // what is this for? + $em = $this->getExpressionManager(); + $status = $em->Evaluate($expr); + $errs = $em->GetReadableErrors(); + $result = $em->GetResult(); + return $result; + } + + private function getReservedWordArray() + { + if (isset($this->knownReservedWords)) { + return $this->knownReservedWords; + } + $this->createVarArrays(); + return $this->knownReservedWords(); + } + + /** + * Return full list of variable names and values. + * TODO: Is there an existing function that does this? + * TODO: Want to only call this once per page refresh + * @return array + / + + private function getVarArray() + { + if (isset($this->knownVars)) { + return $this->knownVars; + } + $this->createVarArrays(); + return $this->knownVars; + } + + private function createVarArrays() + { + $sid = returnglobal('sid'); + $fieldmap=createFieldMap($sid,$style='full'); + $knownVars = array(); // mapping of VarName to Value + $knownSGQAs = array(); // mapping of SGQA to Value + if (isset($fieldmap)) + { + foreach($fieldmap as $fielddata) + { + $value = retrieve_Answer($fielddata['fieldname'], $_SESSION['dateformats']['phpdate']); + $knownVars[$fielddata['title']] = $value; + $knownVars[$fielddata['fieldname']] = $value; + $knownSGQAs['INSERTANS:' . $fielddata['fieldname']] = $value; + } + } + $this->knownVars = $knownVars; + $this->knownReservedWords= $knownSGQAs; + } + + /* + * Goal is to create Expression Manager once per page refresh. + * @return <type> + / + + private function getExpressionManager() + { + if (isset($this->em)) + { + return $this->em; + } + + $em = new ExpressionManager(); + $varArray = $this->getVarArray(); + $em->RegisterVarnames($varArray); + $em->RegisterReservedWords($this->getReservedWordArray()); + $this->em = $em; + return $this->em; + } +} Index: dTexts/dTexts.php --- dTexts/dTexts.php Base (BASE) +++ dTexts/dTexts.php Locally Modified (Based On LOCAL) @@ -18,6 +18,7 @@ { $data=explode(':',$str); $funcName=array_shift($data); + $data = array(implode(':',$data)); try { $func = dTexts::loadFunction($funcName); @@ -39,7 +40,7 @@ / public static function loadFunction($name) { - $name=ucfirst(strtolower($name)); + $name=ucwords($name); $fileName='./classes/dTexts/dFunctions/dFunction'.$name.'.php'; if(!file_exists($fileName)) { Index: eval/ExpressionManager.php --- eval/ExpressionManager.php Locally New +++ eval/ExpressionManager.php Locally New @@ -0,0 +1,1865 @@ +<?php +/** + * Description of ExpressionManager + * (1) Does safe evaluation of PHP expressions. Only registered Functions, Variables, and ReservedWords are allowed. + * (a) Functions include any math, string processing, conditional, formatting, etc. functions + * (b) Variables are typically the question name (question.title) + * (c) ReservedWords are any LimeReplacementField or Token, including all INSERTANS:SGQA codes + * (2) This class can replace LimeSurvey's current process of resolving strings that contain LimeReplacementFields + * (a) String is split by expressions (by curly braces, but safely supporting strings and escaped curly braces) + * (b) Expressions (things surrounded by curly braces) are evaluated - thereby doing LimeReplacementField substitution and/or more complex calculations + * (c) Non-expressions are left intact + * (d) The array of stringParts are re-joined to create the desired final string. + * + * At present, all variables are read-only, but this could be extended to support creation of temporary variables and/or read-write access to registered variables + * + * @author Thomas M. White + / + +class ExpressionManager { + // These three variables are effectively static once constructed + private $sExpressionRegex; + private $asTokenType; + private $sTokenizerRegex; + private $asCategorizeTokensRegex; + private $amValidFunctions; // names and # params of valid functions + private $amVars; // names and values of valid variables + private $amReservedWords; // names and values of valid reserved words + + // Thes variables are used while processing the equation + private $expr; // the source expression + private $tokens; // the list of generated tokens + private $count; // total number of $tokens + private $pos; // position within the $token array while processing equation + private $errs; // array of syntax errors + private $onlyparse; + private $stack; // stack of intermediate results + private $result; // final result of evaluating the expression; + private $evalStatus; // true if $result is a valid result, and there are no serious errors + private $varsUsed; // list of variables referenced in the equation + private $reservedWordsUsed; // list of reserved words used in the equation + + // These variables are only used by sProcessStringContainingExpressions + private $allVarsUsed; // full list of variables used within the string, even if contains multiple expressions + private $allReservedWordsUsed; // full list of reserved words used in the string, even if contains multiple expresions + + function __construct() + { + // List of token-matching regular expressions + $regex_dq_string = '(?<!\\\\)".?(?<!\\\\)"'; + $regex_sq_string = '(?<!\\\\)\'.?(?<!\\\\)\''; + $regex_whitespace = '\s+'; + $regex_lparen = '$'; + $regex_rparen = '$'; + $regex_comma = ','; + $regex_not = '!'; + $regex_inc_dec = '\+\+\|--'; + $regex_binary = '[+/-]'; + $regex_compare = '<=\|<\|>=\|>\|==\|!=\|\ble\b\|\blt\b\|\bge\b\|\bgt\b\|\beq\b\|\bne\b'; + $regex_assign = '=\|\+=\|-=\|\=\|/='; + $regex_sgqa = '[0-9]+X[0-9]+X[0-9]+[A-Z0-9_]'; + $regex_word = '[A-Z][A-Z0-9_]:?[A-Z0-9_]'; + $regex_number = '[0-9]+\.?[0-9]\|\.[0-9]+'; + $regex_andor = '\band\b\|\bor\b\|&&\|\\|\\|'; + + $this->sExpressionRegex = '#((?<!\\\\){(' . $regex_dq_string . '\|' . $regex_sq_string . '\|.?)(?<!\\\\)})#'; + + // asTokenRegex and t_tokey_type must be kept in sync (same number and order) + $asTokenRegex = array( + $regex_dq_string, + $regex_sq_string, + $regex_whitespace, + $regex_lparen, + $regex_rparen, + $regex_comma, + $regex_andor, + $regex_compare, + $regex_sgqa, + $regex_word, + $regex_number, + $regex_not, + $regex_inc_dec, + $regex_assign, + $regex_binary, + ); + + $this->asTokenType = array( + 'STRING', + 'STRING', + 'SPACE', + 'LP', + 'RP', + 'COMMA', + 'AND_OR', + 'COMPARE', + 'SGQA', + 'WORD', + 'NUMBER', + 'NOT', + 'OTHER', + 'ASSIGN', + 'BINARYOP', + ); + + // $sTokenizerRegex - a single regex used to split and equation into tokens + $this->sTokenizerRegex = '#(' . implode('\|',$asTokenRegex) . ')#i'; + + // $asCategorizeTokensRegex - an array of patterns so can categorize the type of token found - would be nice if could get this from preg_split + // Adding ability to capture 'OTHER' type, which indicates an error - unsupported syntax element + $this->asCategorizeTokensRegex = preg_replace("#^(.)$#","#^$1$#i",$asTokenRegex); + $this->asCategorizeTokensRegex[] = '/.+/'; + $this->asTokenType[] = 'OTHER'; + + // Each allowed function is a mapping from local name to external name + number of arguments + // Functions can have -1 (meaning unlimited), or a list of serveral allowable #s of arguments. + $this->amValidFunctions = array( + 'abs' =>array('abs','Absolute value',1), + 'acos' =>array('acos','Arc cosine',1), + 'acosh' =>array('acosh','Inverse hyperbolic cosine',1), + 'asin' =>array('asin','Arc sine',1), + 'asinh' =>array('asinh','Inverse hyperbolic sine',1), + 'atan2' =>array('atan2','Arc tangent of two variables',2), + 'atan' =>array('atan','Arc tangent',1), + 'atanh' =>array('atanh','Inverse hyperbolic tangent',1), + 'base_convert' =>array('base_convert','Convert a number between arbitrary bases',3), + 'bindec' =>array('bindec','Binary to decimal',1), + 'ceil' =>array('ceil','Round fractions up',1), + 'cos' =>array('cos','Cosine',1), + 'cosh' =>array('cosh','Hyperbolic cosine',1), + 'decbin' =>array('decbin','Decimal to binary',1), + 'dechex' =>array('dechex','Decimal to hexadecimal',1), + 'decoct' =>array('decoct','Decimal to octal',1), + 'deg2rad' =>array('deg2rad','Converts the number in degrees to the radian equivalent',1), + 'exp' =>array('exp','Calculates the exponent of e',1), + 'expm1' =>array('expm1','Returns exp(number) - 1, computed in a way that is accurate even when the value of number is close to zero',1), + 'floor' =>array('floor','Round fractions down',1), + 'fmod' =>array('fmod','Returns the floating point remainder (modulo) of the division of the arguments',2), + 'getrandmax' =>array('getrandmax','Show largest possible random value',0), + 'hexdec' =>array('hexdec','Hexadecimal to decimal',1), + 'hypot' =>array('hypot','Calculate the length of the hypotenuse of a right-angle triangle',2), + 'is_finite' =>array('is_finite','Finds whether a value is a legal finite number',1), + 'is_infinite' =>array('is_infinite','Finds whether a value is infinite',1), + 'is_nan' =>array('is_nan','Finds whether a value is not a number',1), + 'lcg_value' =>array('lcg_value','Combined linear congruential generator',0), + 'log10' =>array('log10','Base-10 logarithm',1), + 'log1p' =>array('log1p','Returns log(1 + number), computed in a way that is accurate even when the value of number is close to zero',1), + 'log' =>array('log','Natural logarithm',1,2), + 'max' =>array('max','Find highest value',-1), + 'min' =>array('min','Find lowest value',-1), + 'mt_getrandmax' =>array('mt_getrandmax','Show largest possible random value',0), + 'mt_rand' =>array('mt_rand','Generate a better random value',0,2), + 'mt_srand' =>array('mt_srand','Seed the better random number generator',0,1), + 'octdec' =>array('octdec','Octal to decimal',1), + 'pi' =>array('pi','Get value of pi',0), + 'pow' =>array('pow','Exponential expression',2), + 'rad2deg' =>array('rad2deg','Converts the radian number to the equivalent number in degrees',1), + 'rand' =>array('rand','Generate a random integer',0,2), + 'round' =>array('round','Rounds a float',1,2,3), + 'sin' =>array('sin','Sine',1), + 'sinh' =>array('sinh','Hyperbolic sine',1), + 'sqrt' =>array('sqrt','Square root',1), + 'srand' =>array('srand','Seed the random number generator',0,1), + 'sum' =>array('array_sum','Calculate the sum of values in an array',-1), + 'tan' =>array('tan','Tangent',1), + 'tanh' =>array('tanh','Hyperbolic tangent',1), + + 'empty' =>array('empty','Determine whether a variable is empty',1), + 'intval' =>array('intval','Get the integer value of a variable',1,2), + 'is_bool' =>array('is_bool','Finds out whether a variable is a boolean',1), + 'is_float' =>array('is_float','Finds whether the type of a variable is float',1), + 'is_int' =>array('is_int','Find whether the type of a variable is integer',1), + 'is_null' =>array('is_null','Finds whether a variable is NULL',1), + 'is_numeric' =>array('is_numeric','Finds whether a variable is a number or a numeric string',1), + 'is_scalar' =>array('is_scalar','Finds whether a variable is a scalar',1), + 'is_string' =>array('is_string','Find whether the type of a variable is string',1), + + 'addcslashes' =>array('addcslashes','Quote string with slashes in a C style',2), + 'addslashes' =>array('addslashes','Quote string with slashes',1), + 'bin2hex' =>array('bin2hex','Convert binary data into hexadecimal representation',1), + 'chr' =>array('chr','Return a specific character',1), + 'chunk_split' =>array('chunk_split','Split a string into smaller chunks',1,2,3), + 'convert_uudecode' =>array('convert_uudecode','Decode a uuencoded string',1), + 'convert_uuencode' =>array('convert_uuencode','Uuencode a string',1), + 'count_chars' =>array('count_chars','Return information about characters used in a string',1,2), + 'crc32' =>array('crc32','Calculates the crc32 polynomial of a string',1), + 'crypt' =>array('crypt','One-way string hashing',1,2), + 'hebrev' =>array('hebrev','Convert logical Hebrew text to visual text',1,2), + 'hebrevc' =>array('hebrevc','Convert logical Hebrew text to visual text with newline conversion',1,2), + 'html_entity_decode' =>array('html_entity_decode','Convert all HTML entities to their applicable characters',1,2,3), + 'htmlentities' =>array('htmlentities','Convert all applicable characters to HTML entities',1,2,3), + 'htmlspecialchars_decode' =>array('htmlspecialchars_decode','Convert special HTML entities back to characters',1,2), + 'htmlspecialchars' =>array('htmlspecialchars','Convert special characters to HTML entities',1,2,3,4), + 'implode' =>array('implode','Join array elements with a string',-1), + 'lcfirst' =>array('lcfirst','Make a string\'s first character lowercase',1), + 'levenshtein' =>array('levenshtein','Calculate Levenshtein distance between two strings',2,5), + 'ltrim' =>array('ltrim','Strip whitespace (or other characters) from the beginning of a string',1,2), + 'md5' =>array('md5','Calculate the md5 hash of a string',1), + 'metaphone' =>array('metaphone','Calculate the metaphone key of a string',1,2), + 'money_format' =>array('money_format','Formats a number as a currency string',1,2), + 'nl2br' =>array('nl2br','Inserts HTML line breaks before all newlines in a string',1,2), + 'number_format' =>array('number_format','Format a number with grouped thousands',1,2,4), + 'ord' =>array('ord','Return ASCII value of character',1), + 'quoted_printable_decode' =>array('quoted_printable_decode','Convert a quoted-printable string to an 8 bit string',1), + 'quoted_printable_encode' =>array('quoted_printable_encode','Convert a 8 bit string to a quoted-printable string',1), + 'quotemeta' =>array('quotemeta','Quote meta characters',1), + 'rtrim' =>array('rtrim','Strip whitespace (or other characters) from the end of a string',1,2), + 'sha1' =>array('sha1','Calculate the sha1 hash of a string',1), + 'similar_text' =>array('similar_text','Calculate the similarity between two strings',1,2), + 'soundex' =>array('soundex','Calculate the soundex key of a string',1), + 'sprintf' =>array('sprintf','Return a formatted string',-1), + 'str_ireplace' =>array('str_ireplace','Case-insensitive version of str_replace',3), + 'str_pad' =>array('str_pad','Pad a string to a certain length with another string',2,3,4), + 'str_repeat' =>array('str_repeat','Repeat a string',2), + 'str_replace' =>array('str_replace','Replace all occurrences of the search string with the replacement string',3), + 'str_rot13' =>array('str_rot13','Perform the rot13 transform on a string',1), + 'str_shuffle' =>array('str_shuffle','Randomly shuffles a string',1), + 'str_word_count' =>array('str_word_count','Return information about words used in a string',1), + 'strcasecmp' =>array('strcasecmp','Binary safe case-insensitive string comparison',2), + 'strcmp' =>array('strcmp','Binary safe string comparison',2), + 'strcoll' =>array('strcoll','Locale based string comparison',2), + 'strcspn' =>array('strcspn','Find length of initial segment not matching mask',2,3,4), + 'strip_tags' =>array('strip_tags','Strip HTML and PHP tags from a string',1,2), + 'stripcslashes' =>array('stripcslashes','Un-quote string quoted with addcslashes',1), + 'stripos' =>array('stripos','Find position of first occurrence of a case-insensitive string',2,3), + 'stripslashes' =>array('stripslashes','Un-quotes a quoted string',1), + 'stristr' =>array('stristr','Case-insensitive strstr',2,3), + 'strlen' =>array('strlen','Get string length',1), + 'strnatcasecmp' =>array('strnatcasecmp','Case insensitive string comparisons using a "natural order" algorithm',2), + 'strnatcmp' =>array('strnatcmp','String comparisons using a "natural order" algorithm',2), + 'strncasecmp' =>array('strncasecmp','Binary safe case-insensitive string comparison of the first n characters',3), + 'strncmp' =>array('strncmp','Binary safe string comparison of the first n characters',3), + 'strpbrk' =>array('strpbrk','Search a string for any of a set of characters',2), + 'strpos' =>array('strpos','Find position of first occurrence of a string',2,3), + 'strrchr' =>array('strrchr','Find the last occurrence of a character in a string',2), + 'strrev' =>array('strrev','Reverse a string',1), + 'strripos' =>array('strripos','Find position of last occurrence of a case-insensitive string in a string',2,3), + 'strrpos' =>array('strrpos','Find the position of the last occurrence of a substring in a string',2,3), + 'strspn' =>array('Finds the length of the initial segment of a string consisting entirely of characters contained within a given mask.',2,3,4), + 'strstr' =>array('strstr','Find first occurrence of a string',2,3), + 'strtolower' =>array('strtolower','Make a string lowercase',1), + 'strtoupper' =>array('strtoupper','Make a string uppercase',1), + 'strtr' =>array('strtr','Translate characters or replace substrings',3), + 'substr_compare'=>array('substr_compare','Binary safe comparison of two strings from an offset, up to length characters',3,4,5), + 'substr_count' =>array('substr_count','Count the number of substring occurrences',2,3,4), + 'substr_replace'=>array('substr_replace','Replace text within a portion of a string',3,4), + 'substr' =>array('substr','Return part of a string',2,3), + 'ucfirst' =>array('ucfirst','Make a string\'s first character uppercase',1), + 'ucwords' =>array('ucwords','Uppercase the first character of each word in a string',1), + + 'stddev' =>array('stats_standard_deviation','Returns the standard deviation',-1), + + // Locally declared functions + 'if' => array('exprmgr_if','Excel-style if(test,result_if_true,result_if_false)',3), + 'list' => array('exprmgr_list','Return comma-separated list of values',-1), + ); + + $this->amVars = array(); + $this->amReservedWords = array(); + + } + + /** + * Add an error to the error log + * + * @param <type> $errMsg + * @param <type> $token + / + private function AddError($errMsg, $token) + { + $this->errs[] = array($errMsg, $token); + } + + /* + * EvalBinary() computes binary expressions, such as (a or b), (c * d), popping the top two entries off the + * stack and pushing the result back onto the stack. + * + * @param array $token + * @return boolean - false if there is any error, else true + / + + private function EvalBinary(array $token) + { + if (count($this->stack) < 2) + { + $this->AddError("Unable to evaluate binary operator - fewer than 2 entries on stack", $token); + return false; + } + $arg2 = $this->StackPop(); + $arg1 = $this->StackPop(); + if (is_null($arg1) or is_null($arg2)) + { + $this->AddError("Invalid value(s) on the stack", $token); + return false; + } + // TODO: try to determine datatype? + switch(strtolower($token[0])) + { + case 'or': + case '\|\|': + $result = array(($arg1[0] or $arg2[0]),$token[1],'NUMBER'); + break; + case 'and': + case '&&': + $result = array(($arg1[0] and $arg2[0]),$token[1],'NUMBER'); + break; + case '==': + case 'eq': + $result = array(($arg1[0] == $arg2[0]),$token[1],'NUMBER'); + break; + case '!=': + case 'ne': + $result = array(($arg1[0] != $arg2[0]),$token[1],'NUMBER'); + break; + case '<': + case 'lt': + $result = array(($arg1[0] < $arg2[0]),$token[1],'NUMBER'); + break; + case '<='; + case 'le': + $result = array(($arg1[0] <= $arg2[0]),$token[1],'NUMBER'); + break; + case '>': + case 'gt': + $result = array(($arg1[0] > $arg2[0]),$token[1],'NUMBER'); + break; + case '>='; + case 'ge': + $result = array(($arg1[0] >= $arg2[0]),$token[1],'NUMBER'); + break; + case '+': + $result = array(($arg1[0] + $arg2[0]),$token[1],'NUMBER'); + break; + case '-': + $result = array(($arg1[0] - $arg2[0]),$token[1],'NUMBER'); + break; + case '': + $result = array(($arg1[0] * $arg2[0]),$token[1],'NUMBER'); + break; + case '/'; + $result = array(($arg1[0] / $arg2[0]),$token[1],'NUMBER'); + break; + } + $this->StackPush($result); + return true; + } + + /** + * Processes operations like +a, -b, !c + * @param array $token + * @return boolean - true if success, false if any error occurred + / + + private function EvalUnary(array $token) + { + if (count($this->stack) < 1) + { + $this->AddError("Unable to evaluate unary operator - no entries on stack", $token); + return false; + } + $arg1 = $this->StackPop(); + if (is_null($arg1)) + { + $this->AddError("Invalid value(s) on the stack", $token); + return false; + } + // TODO: try to determine datatype? + switch($token[0]) + { + case '+': + $result = array((+$arg1[0]),$token[1],'NUMBER'); + break; + case '-': + $result = array((-$arg1[0]),$token[1],'NUMBER'); + break; + case '!'; + $result = array((!$arg[0]),$token[1],'NUMBER'); + break; + } + $this->StackPush($result); + return true; + } + + + /* + * Main entry function + * @param <type> $expr + * @param <type> $onlyparse - if true, then validate the syntax without computing an answer + * @return boolean - true if success, false if any error occurred + / + + public function Evaluate($expr, $onlyparse=false) + { + $this->expr = $expr; + $this->tokens = $this->amTokenize($expr); + $this->count = count($this->tokens); + $this->pos = -1; // starting position within array (first act will be to increment it) + $this->errs = array(); + $this->onlyparse = $onlyparse; + $this->stack = array(); + $this->evalStatus = false; + $this->result = NULL; + $this->varsUsed = array(); + $this->reservedWordsUsed = array(); + + if ($this->HasSyntaxErrors()) { + return false; + } + else if ($this->EvaluateExpressions()) + { + if ($this->pos < $this->count) + { + $this->AddError("Extra tokens found starting at", $this->tokens[$this->pos]); + return false; + } + $this->result = $this->StackPop(); + if (is_null($this->result)) + { + return false; + } + if (count($this->stack) == 0) + { + $this->evalStatus = true; + return true; + } + else + { + $this-AddError("Unbalanced equation - values left on stack",NULL); + return false; + } + } + else + { + $this->AddError("Not a valid expression",NULL); + return false; + } + } + + + /* + * Process "a op b" where op in (+,-,concatenate) + * @return boolean - true if success, false if any error occurred + / + private function EvaluateAdditiveExpression() + { + if (!$this->EvaluateMultiplicativeExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '+': + case '-'; + if ($this->EvaluateMultiplicativeExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue; + } + else + { + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process a Constant (number of string), retrieve the value of a known variable, or process a function, returning result on the stack. + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateConstantVarOrFunction() + { + if ($this->pos + 1 >= $this->count) + { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + switch ($token[2]) + { + case 'NUMBER': + case 'STRING': + $this->StackPush($token); + return true; + break; + case 'WORD': + case 'SGQA': + if (($this->pos + 1) < $this->count and $this->tokens[($this->pos + 1)][2] == 'LP') + { + return $this->EvaluateFunction(); + } + else + { + if ($this->isValidVariable($token[0])) + { + $this->varsUsed[] = $token[0]; // add this variable to list of those used in this equation + $result = array($this->amVars[$token[0]],$token[1],'NUMBER'); + $this->StackPush($result); + return true; + } + else if ($this->isValidReservedWord($token[0])) + { + $this->reservedWordsUsed[] = $token[0]; + $result = array($this->amReservedWords[$token[0]],$token[1],'NUMBER'); + $this->StackPush($result); + return true; + } + else + { + $this->AddError("Undefined variable or reserved word", $token); + return false; + } + } + break; + case 'COMMA': + --$this->pos; + $this->AddError("Should never get to this line?",$token); + return false; + default: + return false; + break; + } + } + + /* + * Process "a == b", "a eq b", "a != b", "a ne b" + * @return boolean - true if success, false if any error occurred + / + private function EvaluateEqualityExpression() + { + if (!$this->EvaluateRelationExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '==': + case 'eq': + case '!=': + case 'ne': + if ($this->EvaluateRelationExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue; + } + else + { + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process a single expression (e.g. without commas) + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateExpression() + { + if ($this->pos + 2 < $this->count) + { + $token1 = $this->tokens[++$this->pos]; + $token2 = $this->tokens[++$this->pos]; + if ($this->isValidVariable($token1[0]) and $token2[2] == 'ASSIGN') + { + $evalStatus = $this->EvaluateLogicalOrExpression(); + if ($evalStatus) + { + $result = $this->StackPop(); + if (!is_null($result)) + { + $newResult = $token2; + $newResult[2] = 'NUMBER'; + $newResult[0] = $this->setVariableValue($token2[0], $token1[0], $result[0]); + $this->StackPush($newResult); + } + else + { + $evalStatus = false; + } + } + return $evalStatus; + } + else + { + // not an assignment expression, so try something else + $this->pos -= 2; + return $this->EvaluateLogicalOrExpression(); + } + } + else + { + return $this->EvaluateLogicalOrExpression(); + } + } + + /* + * Process "expression [, expression]* + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateExpressions() + { + $evalStatus = $this->EvaluateExpression(); + if (!$evalStatus) + { + return false; + } + + while (++$this->pos < $this->count) { + $token = $this->tokens[$this->pos]; + if ($token[2] == 'RP') + { + return true; // presumbably the end of an expression + } + else if ($token[2] == 'COMMA') + { + if ($this->EvaluateExpression()) + { + $secondResult = $this->StackPop(); + $firstResult = $this->StackPop(); + if (is_null($firstResult)) + { + return false; + } + $this->StackPush($secondResult); + $evalStatus = true; + } + + } + else + { + $this->AddError("Expected expressions separated by commas",$token); + $evalStatus = false; + break; + } + } + while (++$this->pos < $this->count) + { + $token = $this->tokens[$this->pos]; + $this->AddError("Extra token found after Expressions",$token); + $evalStatus = false; + } + return $evalStatus; + } + + /* + * Process a function call + * @return boolean - true if success, false if any error occurred + / + private function EvaluateFunction() + { + $funcNameToken = $this->tokens[$this->pos]; // note that don't need to increment position for functions + $funcName = $funcNameToken[0]; + if (!$this->isValidFunction($funcName)) + { + $this->AddError("Undefined Function", $funcNameToken); + return false; + } + $token2 = $this->tokens[++$this->pos]; + if ($token2[2] != 'LP') + { + $this->AddError("Expected '(' after function name", $token); + } + $params = array(); // will just store array of values, not tokens + while ($this->pos + 1 < $this->count) + { + $token3 = $this->tokens[$this->pos + 1]; + if (count($params) > 0) + { + // should have COMMA or RP + if ($token3[2] == 'COMMA') + { + ++$this->pos; // consume the token so can process next clause + if ($this->EvaluateExpression()) + { + $value = $this->StackPop(); + if (is_null($value)) + { + return false; + } + $params[] = $value[0]; + continue; + } + else + { + $this->AddError("Extra comma found in function", $token3); + return false; + } + } + } + if ($token3[2] == 'RP') + { + ++$this->pos; // consume the token so can process next clause + return $this->RunFunction($funcNameToken,$params); + } + else + { + if ($this->EvaluateExpression()) + { + $value = $this->StackPop(); + if (is_null($value)) + { + return false; + } + $params[] = $value[0]; + continue; + } + else + { + return false; + } + } + } + } + + /* + * Process "a && b" or "a and b" + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateLogicalAndExpression() + { + if (!$this->EvaluateEqualityExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '&&': + case 'and': + if ($this->EvaluateEqualityExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process "a \|\| b" or "a or b" + * @return boolean - true if success, false if any error occurred + / + private function EvaluateLogicalOrExpression() + { + if (!$this->EvaluateLogicalAndExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '\|\|': + case 'or': + if ($this->EvaluateLogicalAndExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + default: + // no more expressions being ORed together, so continue parsing + --$this->pos; + return true; + } + } + // no more tokens to parse + return true; + } + + /* + * Process "a op b" where op in (,/) + @return boolean - true if success, false if any error occurred + / + + private function EvaluateMultiplicativeExpression() + { + if (!$this->EvaluateUnaryExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '': + case '/'; + if ($this->EvaluateUnaryExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /** + * Process expressions including functions and parenthesized blocks + * @return boolean - true if success, false if any error occurred + / + + private function EvaluatePrimaryExpression() + { + if (($this->pos + 1) >= $this->count) { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + if ($token[2] == 'LP') + { + if (!$this->EvaluateExpressions()) + { + return false; + } + $token = $this->tokens[$this->pos]; + if ($token[2] == 'RP') + { + return true; + } + else + { + $this->AddError("Expected ')'", $token); + return false; + } + } + else + { + --$this->pos; + return $this->EvaluateConstantVarOrFunction(); + } + } + + /* + * Process "a op b" where op in (lt, gt, le, ge, <, >, <=, >=) + * @return boolean - true if success, false if any error occurred + / + private function EvaluateRelationExpression() + { + if (!$this->EvaluateAdditiveExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '<': + case 'lt': + case '<='; + case 'le': + case '>': + case 'gt': + case '>='; + case 'ge': + if ($this->EvaluateAdditiveExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process "op a" where op in (+,-,!) + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateUnaryExpression() + { + if (($this->pos + 1) >= $this->count) { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '+': + case '-': + case '!': + if (!$this->EvaluatePrimaryExpression()) + { + return false; + } + return $this->EvalUnary($token); + break; + default: + --$this->pos; + return $this->EvaluatePrimaryExpression(); + } + } + + /* + * Returns array of all reserved words used when parsing a string via sProcessStringContainingExpressions + * @return <type> + / + + public function GetAllReservedWordsUsed() + { + return array_unique($this->allReservedWordsUsed); + } + + /* + * Returns array of all variables used when parsing a string via sProcessStringContainingExpressions + * @return <type> + / + public function GetAllVarsUsed() + { + return array_unique($this->allVarsUsed); + } + + /* + * Return the result of evaluating the equation - NULL if error + * @return mixed + / + public function GetResult() + { + return $this->result[0]; + } + + /* + * Return an array of errors + * @return array + / + public function GetErrors() + { + return $this->errs; + } + + /* + * Return an array of human-readable errors (message, offending token, offset of offending token within equation) + * @return array + / + public function GetReadableErrors() + { + $errs = array(); + foreach ($this->errs as $err) + { + $msg = $err[0]; + $token = $err[1]; + $toshow = 'ERR'; + if (!is_null($token)) + { + $toshow .= '[' . $token[0] . ' @pos=' . $token[1] . ']'; + } + $toshow .= ': ' . $msg; + $errs[] = $toshow; + } + return $errs; + } + + /* + * Return array of list of reserved words used in the equation + * @return <type> + / + + public function GetReservedWordsUsed() + { + return array_unique($this->reservedWordsUsed); + } + + /* + * Return array of the list of variables used in the equation + * @return array + / + public function GetVarsUsed() + { + return array_unique($this->varsUsed); + } + + /* + * Return true if there were syntax or processing errors + * @return boolean + / + public function HasErrors() + { + return (count($this->errs) > 0); + } + + /* + * Return true if there are syntax errors + * @return boolean + / + private function HasSyntaxErrors() + { + // check for bad tokens + // check for unmatched parentheses + // check for undefined variables + // check for undefined functions (but can't easily check allowable # elements?) + + $nesting = 0; + + for ($i=0;$i<$this->count;++$i) + { + $token = $this->tokens[$i]; + switch ($token[2]) + { + case 'LP': + ++$nesting; + break; + case 'RP': + --$nesting; + if ($nesting < 0) + { + $this->AddError("Extra ')' detected", $token); + } + break; + case 'WORD': + case 'SGQA': + if ($i+1 < $this->count and $this->tokens[$i+1][2] == 'LP') + { + if (!$this->isValidFunction($token[0])) + { + $this->AddError("Undefined function", $token); + } + } + else + { + if (!($this->isValidVariable($token[0]) or $this->isValidReservedWord($token[0]))) + { + $this->AddError("Undefined variable or reserved word", $token); + } + } + break; + case 'OTHER': + $this->AddError("Unsupported syntax", $token); + break; + default: + break; + } + } + if ($nesting != 0) + { + $this->AddError("Parentheses not balanced",NULL); + } + return (count($this->errs) > 0); + } + + /* + * Return true if the function name is registered + * @param <type> $name + * @return boolean + / + + private function isValidFunction($name) + { + return array_key_exists($name,$this->amValidFunctions); + } + + /* + * Return true if the reserved word name is registered + * @param <type> $name + * @return boolean + / + private function isValidReservedWord($name) + { + return array_key_exists($name,$this->amReservedWords); + } + + /* + * Return true if the variable name is registered + * @param <type> $name + * @return boolean + / + private function isValidVariable($name) + { + return array_key_exists($name,$this->amVars); + } + + /* + * Process a full string, containing multiple expressions delimited by {}, return a consolidated string + * @param <type> $src + / + + public function sProcessStringContainingExpressions($src) + { + // tokenize string by the {} pattern, properly dealing with strings in quotations, and escaped curly brace values + $stringParts = $this->asSplitStringOnExpressions($src); + + $resolvedParts = array(); + $this->allVarsUsed = array(); + $this->allReservedWordsUsed = array(); + + foreach ($stringParts as $stringPart) + { + if ($stringPart[2] == 'STRING') { + $resolvedParts[] = $stringPart[0]; + } + else { + if ($this->Evaluate(substr($stringPart[0],1,-1))) + { + $resolvedParts[] = $this->GetResult(); + $this->allVarsUsed = array_merge($this->allVarsUsed,$this->GetVarsUsed()); + $this->allReservedWordsUsed = array_merge($this->allReservedWordsUsed, $this->GetReservedWordsUsed()); + } + else + { + // show original and errors in-line? + $resolvedParts[] = '[' . $stringPart[0] . ':' . implode(';',$this->GetReadableErrors()) . ']'; + } + } + } + $result = implode('',$resolvedParts); + return $result; + } + + /* + * Run a registered function + * @param <type> $funcNameToken + * @param <type> $params + * @return boolean + / + private function RunFunction($funcNameToken,$params) + { + $name = $funcNameToken[0]; + if (!$this->isValidFunction($name)) + { + return false; + } + $func = $this->amValidFunctions[$name]; + $funcName = $func[0]; + $numArgs = count($params); + + if (function_exists($funcName)) { + $numArgsAllowed = array_slice($func, 2); + $argsPassed = is_array($params) ? count($params) : 0; + + // for unlimited # parameters + try + { + if (in_array(-1, $numArgsAllowed)) { + $result = $funcName($params); + + // Call function with the params passed + } elseif (in_array($argsPassed, $numArgsAllowed)) { + + switch ($argsPassed) { + case 0: + $result = $funcName(); + break; + case 1: + $result = $funcName($params[0]); + break; + case 2: + $result = $funcName($params[0], $params[1]); + break; + case 3: + $result = $funcName($params[0], $params[1], $params[2]); + break; + case 4: + $result = $funcName($params[0], $params[1], $params[2], $params[3]); + break; + default: + $this->AddError("Error: Unsupported arg count: $funcName(".implode(", ",$params),$funcNameToken); + return false; + } + + } else { + $this->AddError("Error: Incorrect arg count: " . $funcName ."(".implode(", ",$params).")",$funcNameToken); + return false; + } + } + catch (Exception $e) + { + $this->AddError($e->getMessage(),$funcNameToken); + return false; + } + $token = array($result,$funcNameToken[1],'NUMBER'); + $this->StackPush($token); + return true; + } + } + + /* + * Add user functions to array of allowable functions within the equation. + * $functions is an array of key to value mappings like this: + * 'newfunc' => array('my_func_script', 1,3) + * where 'newfunc' is the name of an allowable function wihtin the expression, 'my_func_script' is the registered PHP function name, + * and 1,3 are the list of allowable numbers of paremeters (so my_func() can take 1 or 3 parameters. + * + * @param array $functions + / + + public function RegisterFunctions(array $functions) { + $this->amValidFunctions= array_merge($this->amValidFunctions, $functions); + } + + /* + * Add list of allowable ReservedWord names within the equation + * $varnames is an array of key to value mappings like this: + * 'myvar' => value + * where value is optional (e.g. can be blank), and can be any scalar type (e.g. string, number, but not array) + * the system will use the values as fast lookup when doing calculations, but if it needs to set values, it will call + * the interface function to set the values by name + * + * @param array $varnames + / + public function RegisterReservedWords(array $varnames) { + $this->amReservedWords = array_merge($this->amReservedWords, $varnames); + } + + /* + * Add list of allowable variable names within the equation + * $varnames is an array of key to value mappings like this: + * 'myvar' => value + * where value is optional (e.g. can be blank), and can be any scalar type (e.g. string, number, but not array) + * the system will use the values as fast lookup when doing calculations, but if it needs to set values, it will call + * the interface function to set the values by name + * + * @param array $varnames + / + public function RegisterVarnames(array $varnames) { + $this->amVars = array_merge($this->amVars, $varnames); + } + + /* + * Set the value of a registered variable + * @param $op - the operator (=,=,/=,+=,-=) + @param <type> $name + * @param <type> $value + / + private function setVariableValue($op,$name,$value) + { + // TODO - set this externally + switch($op) + { + case '=': + $this->amVars[$name] = $value; + break; + case '=': + $this->amVars[$name] = $value; + break; + case '/=': + $this->amVars[$name] /= $value; + break; + case '+=': + $this->amVars[$name] += $value; + break; + case '-=': + $this->amVars[$name] -= $value; + break; + } + return $this->amVars[$name]; + } + + public function asSplitStringOnExpressions($src) + { + // tokenize string by the {} pattern, propertly dealing with strings in quotations, and escaped curly brace values + $tokens0 = preg_split($this->sExpressionRegex,$src,-1,(PREG_SPLIT_NO_EMPTY \| PREG_SPLIT_DELIM_CAPTURE \| PREG_SPLIT_OFFSET_CAPTURE)); + + $tokens = array(); + // Add token_type to $tokens: For each token, test each categorization in order - first match will be the best. + for ($j=0;$j<count($tokens0);++$j) + { + $token = $tokens0[$j]; + if (preg_match($this->sExpressionRegex,$token[0])) + { + $token[2] = 'EXPRESSION'; + } + else + { + $token[2] = 'STRING'; + } + $tokens[] = $token; + } + return $tokens; + } + + /* + * Pop a value token off of the stack + * @return token + / + + private function StackPop() + { + if (count($this->stack) > 0) + { + return array_pop($this->stack); + } + else + { + $this->AddError("Tried to pop value off of empty stack", NULL); + return NULL; + } + } + + /* + * Stack only holds values (number, string), not operators + * @param array $token + / + + private function StackPush(array $token) + { + if ($this->onlyparse) + { + // If only parsing, still want to validate syntax, so use "1" for all variables + switch($token[2]) + { + case 'STRING': + $this->stack[] = array(1,$token[1],'STRING'); + break; + case 'NUMBER': + default: + $this->stack[] = array(1,$token[1],'NUMBER'); + break; + } + } + else + { + $this->stack[] = $token; + } + } + + /* + * Split the source string into tokens, removing whitespace, and categorizing them by type. + * + * @param $src + * @return array + / + + private function amTokenize($src) + { + // $tokens0 = array of tokens from equation, showing value and offset position. Will include SPACE, which should be removed + $tokens0 = preg_split($this->sTokenizerRegex,$src,-1,(PREG_SPLIT_NO_EMPTY \| PREG_SPLIT_DELIM_CAPTURE \| PREG_SPLIT_OFFSET_CAPTURE)); + + // $tokens = array of tokens from equation, showing value, offsete position, and type. Will not contain SPACE, but will contain OTHER + $tokens = array(); + // Add token_type to $tokens: For each token, test each categorization in order - first match will be the best. + for ($j=0;$j<count($tokens0);++$j) + { + for ($i=0;$i<count($this->asCategorizeTokensRegex);++$i) + { + $token = $tokens0[$j][0]; + if (preg_match($this->asCategorizeTokensRegex[$i],$token)) + { + if ($this->asTokenType[$i] !== 'SPACE') { + $tokens0[$j][2] = $this->asTokenType[$i]; + if ($this->asTokenType[$i] == 'STRING') + { + // remove outside quotes + $unquotedToken = stripslashes(substr($token,1,-1)); + $tokens0[$j][0] = $unquotedToken; + } + $tokens[] = $tokens0[$j]; // get first matching non-SPACE token type and push onto $tokens array + } + break; // only get first matching token type + } + } + } + return $tokens; + } + + /* + * Unit test the asSplitStringOnExpressions() function to ensure that accurately parses out all expressions + * surrounded by curly braces, allowing for strings and escaped curly braces. + / + + static function UnitTestStringSplitter() + { + $tests = <<<EOD +"this is a string that contains {something in curly braces)" +This example has escaped curly braces like \{this is not an equation\} +Should the parser check for unmatched { opening curly braces? +What about for unmatched } closing curly braces? +{ANS:name}, you said that you are {ANS:age} years old, and that you have {ANS:numKids} {EVAL:if((numKids==1),'child','children')} and {ANS:numPets} {EVAL:if((numPets==1),'pet','pets')} running around the house. So, you have {EVAL:numKids + numPets} wild {EVAL:if((numKids + numPets ==1),'beast','beasts')} to chase around every day. +Since you have more {EVAL:if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'children','pets')} than you do {EVAL:if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')}, do you feel that the {EVAL:if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')} are at a disadvantage? +EOD; + + $em = new ExpressionManager(); + + foreach(explode("\n",$tests) as $test) + { + $tokens = $em->asSplitStringOnExpressions($test); + print '<b>' . $test . '</b><hr/>'; + print '<code>'; + print implode("<br/>\n",explode("\n",print_r($tokens,TRUE))); + print '</code><hr/>'; + } + } + + /* + * Unit test the Tokenizer - Tokenize and generate a HTML-compatible print-out of a comprehensive set of test cases + / + + static function UnitTestTokenizer() + { + // Comprehensive test cases for tokenizing + $tests = <<<EOD + String: "Can strings contain embedded \"quoted passages\" (and parenthesis + other characters?)?" + String: "can single quoted strings" . 'contain nested \'quoted sections\'?'; + Parens: upcase('hello'); + Numbers: 42 72.35 -15 +37 42A .5 0.7 + And_Or: (this and that or the other); Sandles, sorting; (a && b \|\| c) + Words: hi there, my name is C3PO! + UnaryOps: ++a, --b !b + BinaryOps: (a + b c / d) + Comparators: > >= < <= == != gt ge lt le eq ne (target large gents built agile less equal) + Assign: = += -= = /= + SGQA: 1X6X12 1X6X12ber1 1X6X12ber1_lab1 3583X84X249 + Errors: Apt # 10C; (2 > 0) ? 'hi' : 'there'; array[30]; >>> <<< / this is not a comment / // neither is this +EOD; + + $em = new ExpressionManager(); + + foreach(explode("\n",$tests) as $test) + { + $tokens = $em->amTokenize($test); + print '<b>' . $test . '</b><hr/>'; + print '<code>'; + print implode("<br/>\n",explode("\n",print_r($tokens,TRUE))); + print '</code><hr/>'; + } + } + + /* + * Unit test the Evaluator, allowing for passing in of extra functions, variables, and tests + * @param array $extraFunctions + * @param array $extraVars + * @param <type> $extraTests + / + + static function UnitTestEvaluator(array $extraFunctions=array(), array $extraVars=array(), $extraTests='1~1') + { + // Some test cases for Evaluator + $vars = array( + 'one' =>1, + 'two' =>2, + 'three' =>3, + 'four' =>4, + 'five' =>5, + 'six' =>6, + 'seven' =>7, + 'eight' =>8, + 'nine' =>9, + 'ten' =>10, + 'eleven' => 11, + 'twelve' => 12, + 'half' =>.5, + 'hi' =>'there', + 'hello' =>"Tom", + 'a' =>0, + 'b' =>0, + 'c' =>0, + 'd' =>0, + '12X34X56' =>5, + '12X3X5lab1_ber' =>10, + ); + + $reservedWord = array( + 'ADMINEMAIL' =>'{ADMINEMAIL}', + 'ADMINNAME' =>'{ADMINNAME}', + 'AID' =>'{AID}', + 'ANSWERSCLEARED' =>'{ANSWERSCLEARED}', + 'ANSWER' =>'{ANSWER}', + 'ASSESSMENTS' =>'{ASSESSMENTS}', + 'ASSESSMENT_CURRENT_TOTAL' =>'{ASSESSMENT_CURRENT_TOTAL}', + 'ASSESSMENT_HEADING' =>'{ASSESSMENT_HEADING}', + 'CHECKJAVASCRIPT' =>'{CHECKJAVASCRIPT}', + 'CLEARALL' =>'{CLEARALL}', + 'CLOSEWINDOW' =>'{CLOSEWINDOW}', + 'COMPLETED' =>'{COMPLETED}', + 'DATESTAMP' =>'{DATESTAMP}', + 'EMAILCOUNT' =>'{EMAILCOUNT}', + 'EMAIL' =>'{EMAIL}', + 'EXPIRY' =>'{EXPIRY}', + 'FIRSTNAME' =>'{FIRSTNAME}', + 'GID' =>'{GID}', + 'GROUPDESCRIPTION' =>'{GROUPDESCRIPTION}', + 'GROUPNAME' =>'{GROUPNAME}', + 'INSERTANS:123X45X67' =>'{INSERTANS:123X45X67}', + 'INSERTANS:123X45X67ber' =>'{INSERTANS:123X45X67ber}', + 'INSERTANS:123X45X67ber_01a' =>'{INSERTANS:123X45X67ber_01a}', + 'LANGUAGECHANGER' =>'{LANGUAGECHANGER}', + 'LANGUAGE' =>'{LANGUAGE}', + 'LANG' =>'{LANG}', + 'LASTNAME' =>'{LASTNAME}', + 'LOADERROR' =>'{LOADERROR}', + 'LOADFORM' =>'{LOADFORM}', + 'LOADHEADING' =>'{LOADHEADING}', + 'LOADMESSAGE' =>'{LOADMESSAGE}', + 'NAME' =>'{NAME}', + 'NAVIGATOR' =>'{NAVIGATOR}', + 'NOSURVEYID' =>'{NOSURVEYID}', + 'NOTEMPTY' =>'{NOTEMPTY}', + 'NULL' =>'{NULL}', + 'NUMBEROFQUESTIONS' =>'{NUMBEROFQUESTIONS}', + 'OPTOUTURL' =>'{OPTOUTURL}', + 'PASSTHRULABEL' =>'{PASSTHRULABEL}', + 'PASSTHRUVALUE' =>'{PASSTHRUVALUE}', + 'PERCENTCOMPLETE' =>'{PERCENTCOMPLETE}', + 'PERC' =>'{PERC}', + 'PRIVACYMESSAGE' =>'{PRIVACYMESSAGE}', + 'PRIVACY' =>'{PRIVACY}', + 'QID' =>'{QID}', + 'QUESTIONHELPPLAINTEXT' =>'{QUESTIONHELPPLAINTEXT}', + 'QUESTIONHELP' =>'{QUESTIONHELP}', + 'QUESTION_CLASS' =>'{QUESTION_CLASS}', + 'QUESTION_CODE' =>'{QUESTION_CODE}', + 'QUESTION_ESSENTIALS' =>'{QUESTION_ESSENTIALS}', + 'QUESTION_FILE_VALID_MESSAGE' =>'{QUESTION_FILE_VALID_MESSAGE}', + 'QUESTION_HELP' =>'{QUESTION_HELP}', + 'QUESTION_INPUT_ERROR_CLASS' =>'{QUESTION_INPUT_ERROR_CLASS}', + 'QUESTION_MANDATORY' =>'{QUESTION_MANDATORY}', + 'QUESTION_MAN_CLASS' =>'{QUESTION_MAN_CLASS}', + 'QUESTION_MAN_MESSAGE' =>'{QUESTION_MAN_MESSAGE}', + 'QUESTION_NUMBER' =>'{QUESTION_NUMBER}', + 'QUESTION_TEXT' =>'{QUESTION_TEXT}', + 'QUESTION_VALID_MESSAGE' =>'{QUESTION_VALID_MESSAGE}', + 'QUESTION' =>'{QUESTION}', + 'REGISTERERROR' =>'{REGISTERERROR}', + 'REGISTERFORM' =>'{REGISTERFORM}', + 'REGISTERMESSAGE1' =>'{REGISTERMESSAGE1}', + 'REGISTERMESSAGE2' =>'{REGISTERMESSAGE2}', + 'RESTART' =>'{RESTART}', + 'RETURNTOSURVEY' =>'{RETURNTOSURVEY}', + 'SAVEALERT' =>'{SAVEALERT}', + 'SAVEDID' =>'{SAVEDID}', + 'SAVEERROR' =>'{SAVEERROR}', + 'SAVEFORM' =>'{SAVEFORM}', + 'SAVEHEADING' =>'{SAVEHEADING}', + 'SAVEMESSAGE' =>'{SAVEMESSAGE}', + 'SAVE' =>'{SAVE}', + 'SGQ' =>'{SGQ}', + 'SID' =>'{SID}', + 'SITENAME' =>'{SITENAME}', + 'SUBMITBUTTON' =>'{SUBMITBUTTON}', + 'SUBMITCOMPLETE' =>'{SUBMITCOMPLETE}', + 'SUBMITREVIEW' =>'{SUBMITREVIEW}', + 'SURVEYCONTACT' =>'{SURVEYCONTACT}', + 'SURVEYDESCRIPTION' =>'{SURVEYDESCRIPTION}', + 'SURVEYFORMAT' =>'{SURVEYFORMAT}', + 'SURVEYLANGAGE' =>'{SURVEYLANGAGE}', + 'SURVEYLISTHEADING' =>'{SURVEYLISTHEADING}', + 'SURVEYLIST' =>'{SURVEYLIST}', + 'SURVEYNAME' =>'{SURVEYNAME}', + 'SURVEYURL' =>'{SURVEYURL}', + 'TEMPLATECSS' =>'{TEMPLATECSS}', + 'TEMPLATEURL' =>'{TEMPLATEURL}', + 'TEXT' =>'{TEXT}', + 'THEREAREXQUESTIONS' =>'{THEREAREXQUESTIONS}', + 'TIME' =>'{TIME}', + 'TOKEN:EMAIL' =>'{TOKEN:EMAIL}', + 'TOKEN:FIRSTNAME' =>'{TOKEN:FIRSTNAME}', + 'TOKEN:LASTNAME' =>'{TOKEN:LASTNAME}', + 'TOKEN:XXX' =>'{TOKEN:XXX}', + 'TOKENCOUNT' =>'{TOKENCOUNT}', + 'TOKEN_COUNTER' =>'{TOKEN_COUNTER}', + 'TOKEN' =>'{TOKEN}', + 'URL' =>'{URL}', + 'WELCOME' =>'{WELCOME}', + ); + + // Syntax for $tests is~ + // expectedResult~expression + // if the expected result is an error, use NULL for the expected result + $tests = <<<EOD +50~12X34X56 12X3X5lab1_ber +3~a=three +3~c=a +12~c=four +15~c+=a +5~c/=a +-1~c-=six +2~max(one,two) +5~max(one,two,three,four,five) +1024~max(one,(twothree),pow(four,five),six) +1~min(one,two,three,four,five) +27~pow(3,3) +5~hypot(three,four) +0~0 +24~one * two * three * four +-4~five - four - three - two +0~two * three - two - two - two +4~two * three - two +3.1415926535898~pi() +1~pi() == pi() * 2 - pi() +1~sin(pi()/2) +1~sin(0.5 * pi()) +1~sin(pi()/2) == sin(.5 * pi()) +105~5 + 1, 7 * 15 +7~7 +15~10 + 5 +24~12 * 2 +10~13 - 3 +3.5~14 / 4 +5~3 + 1 * 2 +1~one +there~hi +6.25~one * two - three / four + five +1~one + hi +1~two > one +1~two gt one +1~three >= two +1~three ge two +0~four < three +0~four lt three +0~four <= three +0~four le three +0~four == three +0~four eq three +1~four != three +0~four ne four +0~one * hi +5~abs(-five) +0~acos(pi()/2) +0~asin(pi()/2) +10~ceil(9.1) +9~floor(9.9) +32767~getrandmax() +0~rand() +15~sum(one,two,three,four,five) +5~intval(5.7) +1~is_float('5.5') +0~is_float('5') +1~is_numeric(five) +0~is_numeric(hi) +1~is_string(hi) +2.4~(one * two) + (three * four) / (five * six) +1~(one * (two + (three - four) + five) / six) +0~one && 0 +0~two and 0 +1~five && 6 +1~seven && eight +1~one or 0 +1~one \|\| 0 +1~(one and 0) \|\| (two and three) +NULL~hi(there); +NULL~(one * two + (three - four) +NULL~(one * two + (three - four))) +NULL~++a +NULL~--b +11~eleven +144~twelve * twelve +4~if(5 > 7,2,4) +there~if((one > two),'hi','there') +64~if((one < two),pow(2,6),pow(6,2)) +1,2,3,4,5~list(one,two,three,min(four,five,six),max(three,four,five)) +11,12~list(eleven,twelve) +{INSERTANS:123X45X67}~INSERTANS:123X45X67 +{QID}~QID +{ASSESSMENT_HEADING}~ASSESSMENT_HEADING +{TOKEN:FIRSTNAME}~TOKEN:FIRSTNAME +{THEREAREXQUESTIONS}~THEREAREXQUESTIONS +EOD; + + $em = new ExpressionManager(); + $em->RegisterVarnames($vars); + $em->RegisterReservedWords($reservedWord); + + if (is_array($extraVars) and count($extraVars) > 0) + { + $em->RegisterVarnames($extraVars); + } + if (is_array($extraFunctions) and count($extraFunctions) > 0) + { + $em->RegisterFunctions($extraFunctions); + } + if (is_string($extraTests)) + { + $tests .= "\n" . $extraTests; + } + + print '<table border="1"><tr><th>Expression</th><th>Result</th><th>Expected</th><th>VarsUsed</th><th>ReservedWordsUsed</th><th>Errors</th></tr>'; + foreach(explode("\n",$tests)as $test) + { + $values = explode("~",$test); + $expectedResult = array_shift($values); + $expr = implode("~",$values); + $resultStatus = 'ok'; + print '<tr><td>' . $expr . "</td>\n"; + $status = $em->Evaluate($expr); + $result = $em->GetResult(); + $valToShow = $result; + if (is_null($result)) { + $valToShow = "NULL"; + } + print '<td>' . $valToShow . "</td>\n"; + if ($valToShow != $expectedResult) + { + $resultStatus = 'error'; + } + print "<td class='" . $resultStatus . "'>" . $expectedResult . "</td>\n"; + $varsUsed = $em->GetVarsUsed(); + if (is_array($varsUsed) and count($varsUsed) > 0) { + print '<td>' . implode(', ', $varsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $reservedWordsUsed = $em->GetReservedWordsUsed(); + if (is_array($reservedWordsUsed) and count($reservedWordsUsed) > 0) { + print '<td>' . implode(', ', $reservedWordsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $errs = $em->GetReadableErrors(); + $errStatus = 'ok'; + if (is_array($errs) and count($errs) > 0) { + if ($expectedResult != "NULL") + { + $errStatus = 'error'; // should have been error free + } + print "<td class='" . $errStatus . "'>" . implode("<br/>\n", $errs) . "</td>\n"; + } + else { + if ($expectedResult == "NULL") + { + $errStatus = 'error'; // should have had errors + } + print "<td class='" . $errStatus . "'> </td>\n"; + } + print '</tr>'; + } + print '</table>'; + } + + static function UnitTestProcessStringContainingExpressions() + { + $vars = array( + 'name' => 'Sergei', + 'age' => 45, + 'numKids' => 2, + 'numPets' => 1, + ); + $reservedWords = array( + 'INSERTANS:61764X1X1' => 'Peter', + 'INSERTANS:61764X1X2' => 27, + 'INSERTANS:61764X1X3' => 1, + 'INSERTANS:61764X1X4' => 8 + ); + + $tests = <<<EOD +{name}, you said that you are {age} years old, and that you have {numKids} {if((numKids==1),'child','children')} and {numPets} {if((numPets==1),'pet','pets')} running around the house. So, you have {numKids + numPets} wild {if((numKids + numPets ==1),'beast','beasts')} to chase around every day. +Since you have more {if((numKids > numPets),'children','pets')} than you do {if((numKids > numPets),'pets','children')}, do you feel that the {if((numKids > numPets),'pets','children')} are at a disadvantage? +{INSERTANS:61764X1X1}, you said that you are {INSERTANS:61764X1X2} years old, and that you have {INSERTANS:61764X1X3} {if((INSERTANS:61764X1X3==1),'child','children')} and {INSERTANS:61764X1X4} {if((INSERTANS:61764X1X4==1),'pet','pets')} running around the house. So, you have {INSERTANS:61764X1X3 + INSERTANS:61764X1X4} wild {if((INSERTANS:61764X1X3 + INSERTANS:61764X1X4 ==1),'beast','beasts')} to chase around every day. +Since you have more {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'children','pets')} than you do {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')}, do you feel that the {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')} are at a disadvantage? +EOD; + + $em = new ExpressionManager(); + $em->RegisterVarnames($vars); + $em->RegisterReservedWords($reservedWords); + + print '<table border="1"><tr><th>Test</th><th>Result</th><th>VarsUsed</th><th>ReservedWordsUsed</th></tr>'; + foreach(explode("\n",$tests) as $test) + { + print "<tr><td>" . $test . "</td>\n"; + print "<td>" . $em->sProcessStringContainingExpressions($test) . "</td>\n"; + $allVarsUsed = $em->getAllVarsUsed(); + if (is_array($allVarsUsed) and count($allVarsUsed) > 0) { + print "<td>" . implode(', ', $allVarsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $allReservedWordsUsed = $em->getAllReservedWordsUsed(); + if (is_array($allReservedWordsUsed) and count($allReservedWordsUsed) > 0) { + print "<td>" . implode(', ', $allReservedWordsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + print "</tr>\n"; + } + print '</table>'; + } +} + +/* + * Extra Functions can go here. TODO: Find good way to inlcude these extra functions externally. + * Tried via ExpressionManagerFunctions, but they weren't properly included in dFunctionEval.php + / + +function exprmgr_if($test,$ok,$error) +{ + if ($test) + { + return $ok; + } + else + { + return $error; + } +} + +function exprmgr_list($args) +{ + return implode(",",$args); +} + + +?> Index: eval/ExpressionManagerFunctions.php --- eval/ExpressionManagerFunctions.php Locally New +++ eval/ExpressionManagerFunctions.php Locally New @@ -0,0 +1,51 @@ +<?php +/* + * Put functions that you want visibile within Expression Manager in this file + * + * @author Thomas M. White + / + +// Each allowed function is a mapping from local name to external name + number of arguments +// Functions can have -1 (meaning unlimited), or a list of serveral allowable #s of arguments. +$exprmgr_functions = array( + 'if' => array('exprmgr_if','Excel-style if(test,result_if_true,result_if_false)',3), + 'list' => array('exprmgr_list','Return comma-separated list of values',-1), +); + +// Extra static variables for unit tests +$exprmgr_extraVars = array( + 'eleven' => 11, + 'twelve' => 12, +); + +// Unit tests of any added functions +$exprmgr_extraTests = <<<EOD +11:eleven +144:twelve twelve +4:if(5 > 7,2,4) +'there':if((one > two),'hi','there') +64:if((one < two),pow(2,6),pow(6,2)) +1,2,3,4,5:list(one,two,three,min(four,five,six),max(three,four,five)) +11,12:list(eleven,twelve) +EOD; + +function exprmgr_if($test,$ok,$error) +{ + if ($test) + { + return $ok; + } + else + { + return $error; + } +} + +function exprmgr_list($args) +{ + return implode(",",$args); +} + + + +?> Index: eval/Test_ExpressionManager_Evaluate.php --- eval/Test_ExpressionManager_Evaluate.php Locally New +++ eval/Test_ExpressionManager_Evaluate.php Locally New @@ -0,0 +1,29 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <style type="text/css"> + <!-- +.error { + background-color: #ff0000; +} +.ok { + background-color: #00ff00 +} + --> + </style> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; +// include 'ExpressionManagerFunctions.php'; +// ExpressionManager::UnitTestEvaluator($exprmgr_functions,$exprmgr_extraVars,$exprmgr_extraTests); + ExpressionManager::UnitTestEvaluator(); + ?> + </body> +</html> Index: eval/Test_ExpressionManager_ProcessStringContainingExpressions.php --- eval/Test_ExpressionManager_ProcessStringContainingExpressions.php Locally New +++ eval/Test_ExpressionManager_ProcessStringContainingExpressions.php Locally New @@ -0,0 +1,17 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; + ExpressionManager::UnitTestProcessStringContainingExpressions(); + ?> + </body> +</html> Index: eval/Test_ExpressionManager_Tokenizer.php --- eval/Test_ExpressionManager_Tokenizer.php Locally New +++ eval/Test_ExpressionManager_Tokenizer.php Locally New @@ -0,0 +1,18 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; + ExpressionManager::UnitTestTokenizer(); + ExpressionManager::UnitTestStringSplitter(); + ?> + </body> +</html> issue_05103-rev2.patch (83,074 bytes)

TMSWhite 2011-06-12 09:40 reporter ~15412 Last edited: 2011-06-12 09:44 View 2 revisions	I've uploaded a new patch that provides significant enhancements. Description of ExpressionManager (1) Does safe evaluation of PHP expressions. Only registered Functions, Variables, and ReservedWords are allowed. (a) Functions include any math, string processing, conditional, formatting, etc. functions (b) Variables are typically the question name (question.title) (c) ReservedWords are any LimeReplacementField or Token, including all INSERTANS:SGQA codes (2) This class can replace LimeSurvey's current process of resolving strings that contain LimeReplacementFields (a) String is split by expressions (by curly braces, but safely supporting strings and escaped curly braces) (b) Expressions (things surrounded by curly braces) are evaluated - thereby doing LimeReplacementField substitution and/or more complex calculations (c) Non-expressions are left intact (d) The array of stringParts are re-joined to create the desired final string. Try this link to test the new functionality: http://localhost/limesurvey/classes/eval/Test_ExpressionManager_ProcessStringContainingExpressions.php You can also try this new survey: limesurvey_survey_showing_conditional_tailoring_using_equation_parser-rev2.lss Both validate that the parser can now handle SGQA names (and other LimeReplacementFields that contain a colon, like TOKEN:NAME)

TMSWhite 2011-06-12 09:44 reporter	limesurvey_survey_showing_conditional_tailoring_using_equation_parser-rev2.lss (30,727 bytes)

TMSWhite 2011-06-13 18:45 reporter ~15419	Alhtough I initially planned to use the dTexts module to call these expressions (so added support for {ANS:xxxx} and {EVAL:xxxx}, it looks as though dTexts is nearly obsolete, so I'll have the ExpressionManager handle both types of replacement directly. ExpressionManager will parse a string, and do replacement on anything found between curly braces. So, rather than {EVAL:a b - c}, we'd do {a b - c}.

TMSWhite 2011-06-16 08:57 reporter	issue_05103-20110616.patch (90,972 bytes) # This patch file was generated by NetBeans IDE # Following Index: paths are relative to: C:\xampp\htdocs\limesurvey # This patch can be applied using context Tools: Patch action on respective folder. # It uses platform neutral UTF-8 encoding and \n newlines. # Above lines and this line are ignored by the patching process. Index: classes/dTexts/dTexts.php --- classes/dTexts/dTexts.php Base (BASE) +++ classes/dTexts/dTexts.php Locally Modified (Based On LOCAL) @@ -12,12 +12,14 @@ / public static function run($text) { + return $text; // disable this function include_once('dFunctions/dFunctionInterface.php'); preg_match_all('\|\{([^{}]+)\}\|i',$text,$functions); foreach($functions[1] as $id=>$str) { $data=explode(':',$str); $funcName=array_shift($data); + $data = array(implode(':',$data)); try { $func = dTexts::loadFunction($funcName); @@ -39,7 +41,7 @@ / public static function loadFunction($name) { - $name=ucfirst(strtolower($name)); + $name=ucwords($name); $fileName='./classes/dTexts/dFunctions/dFunction'.$name.'.php'; if(!file_exists($fileName)) { Index: classes/eval/ExpressionManager.php --- classes/eval/ExpressionManager.php Locally New +++ classes/eval/ExpressionManager.php Locally New @@ -0,0 +1,1883 @@ +<?php +/** + * Description of ExpressionManager + * (1) Does safe evaluation of PHP expressions. Only registered Functions, Variables, and ReservedWords are allowed. + * (a) Functions include any math, string processing, conditional, formatting, etc. functions + * (b) Variables are typically the question name (question.title) + * (c) ReservedWords are any LimeReplacementField or Token, including all INSERTANS:SGQA codes + * (2) This class can replace LimeSurvey's current process of resolving strings that contain LimeReplacementFields + * (a) String is split by expressions (by curly braces, but safely supporting strings and escaped curly braces) + * (b) Expressions (things surrounded by curly braces) are evaluated - thereby doing LimeReplacementField substitution and/or more complex calculations + * (c) Non-expressions are left intact + * (d) The array of stringParts are re-joined to create the desired final string. + * + * At present, all variables are read-only, but this could be extended to support creation of temporary variables and/or read-write access to registered variables + * + * @author Thomas M. White + / + +class ExpressionManager { + // These three variables are effectively static once constructed + private $sExpressionRegex; + private $asTokenType; + private $sTokenizerRegex; + private $asCategorizeTokensRegex; + private $amValidFunctions; // names and # params of valid functions + private $amVars; // names and values of valid variables + private $amReservedWords; // names and values of valid reserved words + + // Thes variables are used while processing the equation + private $expr; // the source expression + private $tokens; // the list of generated tokens + private $count; // total number of $tokens + private $pos; // position within the $token array while processing equation + private $errs; // array of syntax errors + private $onlyparse; + private $stack; // stack of intermediate results + private $result; // final result of evaluating the expression; + private $evalStatus; // true if $result is a valid result, and there are no serious errors + private $varsUsed; // list of variables referenced in the equation + private $reservedWordsUsed; // list of reserved words used in the equation + + // These variables are only used by sProcessStringContainingExpressions + private $allVarsUsed; // full list of variables used within the string, even if contains multiple expressions + private $allReservedWordsUsed; // full list of reserved words used in the string, even if contains multiple expresions + + function __construct() + { + // List of token-matching regular expressions + $regex_dq_string = '(?<!\\\\)".?(?<!\\\\)"'; + $regex_sq_string = '(?<!\\\\)\'.?(?<!\\\\)\''; + $regex_whitespace = '\s+'; + $regex_lparen = '$'; + $regex_rparen = '$'; + $regex_comma = ','; + $regex_not = '!'; + $regex_inc_dec = '\+\+\|--'; + $regex_binary = '[+/-]'; + $regex_compare = '<=\|<\|>=\|>\|==\|!=\|\ble\b\|\blt\b\|\bge\b\|\bgt\b\|\beq\b\|\bne\b'; + $regex_assign = '=\|\+=\|-=\|\=\|/='; + $regex_sgqa = '[0-9]+X[0-9]+X[0-9]+[A-Z0-9_]\#?[12]?'; + $regex_word = '[A-Z][A-Z0-9_]:?[A-Z0-9_]\.?[A-Z0-9_]\.?[A-Z0-9_]\.?[A-Z0-9_]'; + $regex_number = '[0-9]+\.?[0-9]\|\.[0-9]+'; + $regex_andor = '\band\b\|\bor\b\|&&\|\\|\\|'; + + $this->sExpressionRegex = '#((?<!\\\\){(' . $regex_dq_string . '\|' . $regex_sq_string . '\|.?)(?<!\\\\)})#'; + + // asTokenRegex and t_tokey_type must be kept in sync (same number and order) + $asTokenRegex = array( + $regex_dq_string, + $regex_sq_string, + $regex_whitespace, + $regex_lparen, + $regex_rparen, + $regex_comma, + $regex_andor, + $regex_compare, + $regex_sgqa, + $regex_word, + $regex_number, + $regex_not, + $regex_inc_dec, + $regex_assign, + $regex_binary, + ); + + $this->asTokenType = array( + 'STRING', + 'STRING', + 'SPACE', + 'LP', + 'RP', + 'COMMA', + 'AND_OR', + 'COMPARE', + 'SGQA', + 'WORD', + 'NUMBER', + 'NOT', + 'OTHER', + 'ASSIGN', + 'BINARYOP', + ); + + // $sTokenizerRegex - a single regex used to split and equation into tokens + $this->sTokenizerRegex = '#(' . implode('\|',$asTokenRegex) . ')#i'; + + // $asCategorizeTokensRegex - an array of patterns so can categorize the type of token found - would be nice if could get this from preg_split + // Adding ability to capture 'OTHER' type, which indicates an error - unsupported syntax element + $this->asCategorizeTokensRegex = preg_replace("#^(.)$#","#^$1$#i",$asTokenRegex); + $this->asCategorizeTokensRegex[] = '/.+/'; + $this->asTokenType[] = 'OTHER'; + + // Each allowed function is a mapping from local name to external name + number of arguments + // Functions can have -1 (meaning unlimited), or a list of serveral allowable #s of arguments. + $this->amValidFunctions = array( + 'abs' =>array('abs','Absolute value',1), + 'acos' =>array('acos','Arc cosine',1), + 'acosh' =>array('acosh','Inverse hyperbolic cosine',1), + 'asin' =>array('asin','Arc sine',1), + 'asinh' =>array('asinh','Inverse hyperbolic sine',1), + 'atan2' =>array('atan2','Arc tangent of two variables',2), + 'atan' =>array('atan','Arc tangent',1), + 'atanh' =>array('atanh','Inverse hyperbolic tangent',1), + 'base_convert' =>array('base_convert','Convert a number between arbitrary bases',3), + 'bindec' =>array('bindec','Binary to decimal',1), + 'ceil' =>array('ceil','Round fractions up',1), + 'cos' =>array('cos','Cosine',1), + 'cosh' =>array('cosh','Hyperbolic cosine',1), + 'decbin' =>array('decbin','Decimal to binary',1), + 'dechex' =>array('dechex','Decimal to hexadecimal',1), + 'decoct' =>array('decoct','Decimal to octal',1), + 'deg2rad' =>array('deg2rad','Converts the number in degrees to the radian equivalent',1), + 'exp' =>array('exp','Calculates the exponent of e',1), + 'expm1' =>array('expm1','Returns exp(number) - 1, computed in a way that is accurate even when the value of number is close to zero',1), + 'floor' =>array('floor','Round fractions down',1), + 'fmod' =>array('fmod','Returns the floating point remainder (modulo) of the division of the arguments',2), + 'getrandmax' =>array('getrandmax','Show largest possible random value',0), + 'hexdec' =>array('hexdec','Hexadecimal to decimal',1), + 'hypot' =>array('hypot','Calculate the length of the hypotenuse of a right-angle triangle',2), + 'is_finite' =>array('is_finite','Finds whether a value is a legal finite number',1), + 'is_infinite' =>array('is_infinite','Finds whether a value is infinite',1), + 'is_nan' =>array('is_nan','Finds whether a value is not a number',1), + 'lcg_value' =>array('lcg_value','Combined linear congruential generator',0), + 'log10' =>array('log10','Base-10 logarithm',1), + 'log1p' =>array('log1p','Returns log(1 + number), computed in a way that is accurate even when the value of number is close to zero',1), + 'log' =>array('log','Natural logarithm',1,2), + 'max' =>array('max','Find highest value',-1), + 'min' =>array('min','Find lowest value',-1), + 'mt_getrandmax' =>array('mt_getrandmax','Show largest possible random value',0), + 'mt_rand' =>array('mt_rand','Generate a better random value',0,2), + 'mt_srand' =>array('mt_srand','Seed the better random number generator',0,1), + 'octdec' =>array('octdec','Octal to decimal',1), + 'pi' =>array('pi','Get value of pi',0), + 'pow' =>array('pow','Exponential expression',2), + 'rad2deg' =>array('rad2deg','Converts the radian number to the equivalent number in degrees',1), + 'rand' =>array('rand','Generate a random integer',0,2), + 'round' =>array('round','Rounds a float',1,2,3), + 'sin' =>array('sin','Sine',1), + 'sinh' =>array('sinh','Hyperbolic sine',1), + 'sqrt' =>array('sqrt','Square root',1), + 'srand' =>array('srand','Seed the random number generator',0,1), + 'sum' =>array('array_sum','Calculate the sum of values in an array',-1), + 'tan' =>array('tan','Tangent',1), + 'tanh' =>array('tanh','Hyperbolic tangent',1), + + 'empty' =>array('empty','Determine whether a variable is empty',1), + 'intval' =>array('intval','Get the integer value of a variable',1,2), + 'is_bool' =>array('is_bool','Finds out whether a variable is a boolean',1), + 'is_float' =>array('is_float','Finds whether the type of a variable is float',1), + 'is_int' =>array('is_int','Find whether the type of a variable is integer',1), + 'is_null' =>array('is_null','Finds whether a variable is NULL',1), + 'is_numeric' =>array('is_numeric','Finds whether a variable is a number or a numeric string',1), + 'is_scalar' =>array('is_scalar','Finds whether a variable is a scalar',1), + 'is_string' =>array('is_string','Find whether the type of a variable is string',1), + + 'addcslashes' =>array('addcslashes','Quote string with slashes in a C style',2), + 'addslashes' =>array('addslashes','Quote string with slashes',1), + 'bin2hex' =>array('bin2hex','Convert binary data into hexadecimal representation',1), + 'chr' =>array('chr','Return a specific character',1), + 'chunk_split' =>array('chunk_split','Split a string into smaller chunks',1,2,3), + 'convert_uudecode' =>array('convert_uudecode','Decode a uuencoded string',1), + 'convert_uuencode' =>array('convert_uuencode','Uuencode a string',1), + 'count_chars' =>array('count_chars','Return information about characters used in a string',1,2), + 'crc32' =>array('crc32','Calculates the crc32 polynomial of a string',1), + 'crypt' =>array('crypt','One-way string hashing',1,2), + 'hebrev' =>array('hebrev','Convert logical Hebrew text to visual text',1,2), + 'hebrevc' =>array('hebrevc','Convert logical Hebrew text to visual text with newline conversion',1,2), + 'html_entity_decode' =>array('html_entity_decode','Convert all HTML entities to their applicable characters',1,2,3), + 'htmlentities' =>array('htmlentities','Convert all applicable characters to HTML entities',1,2,3), + 'htmlspecialchars_decode' =>array('htmlspecialchars_decode','Convert special HTML entities back to characters',1,2), + 'htmlspecialchars' =>array('htmlspecialchars','Convert special characters to HTML entities',1,2,3,4), + 'implode' =>array('implode','Join array elements with a string',-1), + 'lcfirst' =>array('lcfirst','Make a string\'s first character lowercase',1), + 'levenshtein' =>array('levenshtein','Calculate Levenshtein distance between two strings',2,5), + 'ltrim' =>array('ltrim','Strip whitespace (or other characters) from the beginning of a string',1,2), + 'md5' =>array('md5','Calculate the md5 hash of a string',1), + 'metaphone' =>array('metaphone','Calculate the metaphone key of a string',1,2), + 'money_format' =>array('money_format','Formats a number as a currency string',1,2), + 'nl2br' =>array('nl2br','Inserts HTML line breaks before all newlines in a string',1,2), + 'number_format' =>array('number_format','Format a number with grouped thousands',1,2,4), + 'ord' =>array('ord','Return ASCII value of character',1), + 'quoted_printable_decode' =>array('quoted_printable_decode','Convert a quoted-printable string to an 8 bit string',1), + 'quoted_printable_encode' =>array('quoted_printable_encode','Convert a 8 bit string to a quoted-printable string',1), + 'quotemeta' =>array('quotemeta','Quote meta characters',1), + 'rtrim' =>array('rtrim','Strip whitespace (or other characters) from the end of a string',1,2), + 'sha1' =>array('sha1','Calculate the sha1 hash of a string',1), + 'similar_text' =>array('similar_text','Calculate the similarity between two strings',1,2), + 'soundex' =>array('soundex','Calculate the soundex key of a string',1), + 'sprintf' =>array('sprintf','Return a formatted string',-1), + 'str_ireplace' =>array('str_ireplace','Case-insensitive version of str_replace',3), + 'str_pad' =>array('str_pad','Pad a string to a certain length with another string',2,3,4), + 'str_repeat' =>array('str_repeat','Repeat a string',2), + 'str_replace' =>array('str_replace','Replace all occurrences of the search string with the replacement string',3), + 'str_rot13' =>array('str_rot13','Perform the rot13 transform on a string',1), + 'str_shuffle' =>array('str_shuffle','Randomly shuffles a string',1), + 'str_word_count' =>array('str_word_count','Return information about words used in a string',1), + 'strcasecmp' =>array('strcasecmp','Binary safe case-insensitive string comparison',2), + 'strcmp' =>array('strcmp','Binary safe string comparison',2), + 'strcoll' =>array('strcoll','Locale based string comparison',2), + 'strcspn' =>array('strcspn','Find length of initial segment not matching mask',2,3,4), + 'strip_tags' =>array('strip_tags','Strip HTML and PHP tags from a string',1,2), + 'stripcslashes' =>array('stripcslashes','Un-quote string quoted with addcslashes',1), + 'stripos' =>array('stripos','Find position of first occurrence of a case-insensitive string',2,3), + 'stripslashes' =>array('stripslashes','Un-quotes a quoted string',1), + 'stristr' =>array('stristr','Case-insensitive strstr',2,3), + 'strlen' =>array('strlen','Get string length',1), + 'strnatcasecmp' =>array('strnatcasecmp','Case insensitive string comparisons using a "natural order" algorithm',2), + 'strnatcmp' =>array('strnatcmp','String comparisons using a "natural order" algorithm',2), + 'strncasecmp' =>array('strncasecmp','Binary safe case-insensitive string comparison of the first n characters',3), + 'strncmp' =>array('strncmp','Binary safe string comparison of the first n characters',3), + 'strpbrk' =>array('strpbrk','Search a string for any of a set of characters',2), + 'strpos' =>array('strpos','Find position of first occurrence of a string',2,3), + 'strrchr' =>array('strrchr','Find the last occurrence of a character in a string',2), + 'strrev' =>array('strrev','Reverse a string',1), + 'strripos' =>array('strripos','Find position of last occurrence of a case-insensitive string in a string',2,3), + 'strrpos' =>array('strrpos','Find the position of the last occurrence of a substring in a string',2,3), + 'strspn' =>array('Finds the length of the initial segment of a string consisting entirely of characters contained within a given mask.',2,3,4), + 'strstr' =>array('strstr','Find first occurrence of a string',2,3), + 'strtolower' =>array('strtolower','Make a string lowercase',1), + 'strtoupper' =>array('strtoupper','Make a string uppercase',1), + 'strtr' =>array('strtr','Translate characters or replace substrings',3), + 'substr_compare'=>array('substr_compare','Binary safe comparison of two strings from an offset, up to length characters',3,4,5), + 'substr_count' =>array('substr_count','Count the number of substring occurrences',2,3,4), + 'substr_replace'=>array('substr_replace','Replace text within a portion of a string',3,4), + 'substr' =>array('substr','Return part of a string',2,3), + 'ucfirst' =>array('ucfirst','Make a string\'s first character uppercase',1), + 'ucwords' =>array('ucwords','Uppercase the first character of each word in a string',1), + + 'stddev' =>array('stats_standard_deviation','Returns the standard deviation',-1), + + // Locally declared functions + 'if' => array('exprmgr_if','Excel-style if(test,result_if_true,result_if_false)',3), + 'list' => array('exprmgr_list','Return comma-separated list of values',-1), + ); + + $this->amVars = array(); + $this->amReservedWords = array(); + + } + + /* + * Add an error to the error log + * + * @param <type> $errMsg + * @param <type> $token + / + private function AddError($errMsg, $token) + { + $this->errs[] = array($errMsg, $token); + } + + /* + * EvalBinary() computes binary expressions, such as (a or b), (c * d), popping the top two entries off the + * stack and pushing the result back onto the stack. + * + * @param array $token + * @return boolean - false if there is any error, else true + / + + private function EvalBinary(array $token) + { + if (count($this->stack) < 2) + { + $this->AddError("Unable to evaluate binary operator - fewer than 2 entries on stack", $token); + return false; + } + $arg2 = $this->StackPop(); + $arg1 = $this->StackPop(); + if (is_null($arg1) or is_null($arg2)) + { + $this->AddError("Invalid value(s) on the stack", $token); + return false; + } + // TODO: try to determine datatype? + switch(strtolower($token[0])) + { + case 'or': + case '\|\|': + $result = array(($arg1[0] or $arg2[0]),$token[1],'NUMBER'); + break; + case 'and': + case '&&': + $result = array(($arg1[0] and $arg2[0]),$token[1],'NUMBER'); + break; + case '==': + case 'eq': + $result = array(($arg1[0] == $arg2[0]),$token[1],'NUMBER'); + break; + case '!=': + case 'ne': + $result = array(($arg1[0] != $arg2[0]),$token[1],'NUMBER'); + break; + case '<': + case 'lt': + $result = array(($arg1[0] < $arg2[0]),$token[1],'NUMBER'); + break; + case '<='; + case 'le': + $result = array(($arg1[0] <= $arg2[0]),$token[1],'NUMBER'); + break; + case '>': + case 'gt': + $result = array(($arg1[0] > $arg2[0]),$token[1],'NUMBER'); + break; + case '>='; + case 'ge': + $result = array(($arg1[0] >= $arg2[0]),$token[1],'NUMBER'); + break; + case '+': + $result = array(($arg1[0] + $arg2[0]),$token[1],'NUMBER'); + break; + case '-': + $result = array(($arg1[0] - $arg2[0]),$token[1],'NUMBER'); + break; + case '': + $result = array(($arg1[0] * $arg2[0]),$token[1],'NUMBER'); + break; + case '/'; + $result = array(($arg1[0] / $arg2[0]),$token[1],'NUMBER'); + break; + } + $this->StackPush($result); + return true; + } + + /** + * Processes operations like +a, -b, !c + * @param array $token + * @return boolean - true if success, false if any error occurred + / + + private function EvalUnary(array $token) + { + if (count($this->stack) < 1) + { + $this->AddError("Unable to evaluate unary operator - no entries on stack", $token); + return false; + } + $arg1 = $this->StackPop(); + if (is_null($arg1)) + { + $this->AddError("Invalid value(s) on the stack", $token); + return false; + } + // TODO: try to determine datatype? + switch($token[0]) + { + case '+': + $result = array((+$arg1[0]),$token[1],'NUMBER'); + break; + case '-': + $result = array((-$arg1[0]),$token[1],'NUMBER'); + break; + case '!'; + $result = array((!$arg[0]),$token[1],'NUMBER'); + break; + } + $this->StackPush($result); + return true; + } + + + /* + * Main entry function + * @param <type> $expr + * @param <type> $onlyparse - if true, then validate the syntax without computing an answer + * @return boolean - true if success, false if any error occurred + / + + public function Evaluate($expr, $onlyparse=false) + { + $this->expr = $expr; + $this->tokens = $this->amTokenize($expr); + $this->count = count($this->tokens); + $this->pos = -1; // starting position within array (first act will be to increment it) + $this->errs = array(); + $this->onlyparse = $onlyparse; + $this->stack = array(); + $this->evalStatus = false; + $this->result = NULL; + $this->varsUsed = array(); + $this->reservedWordsUsed = array(); + + if ($this->HasSyntaxErrors()) { + return false; + } + else if ($this->EvaluateExpressions()) + { + if ($this->pos < $this->count) + { + $this->AddError("Extra tokens found starting at", $this->tokens[$this->pos]); + return false; + } + $this->result = $this->StackPop(); + if (is_null($this->result)) + { + return false; + } + if (count($this->stack) == 0) + { + $this->evalStatus = true; + return true; + } + else + { + $this-AddError("Unbalanced equation - values left on stack",NULL); + return false; + } + } + else + { + $this->AddError("Not a valid expression",NULL); + return false; + } + } + + + /* + * Process "a op b" where op in (+,-,concatenate) + * @return boolean - true if success, false if any error occurred + / + private function EvaluateAdditiveExpression() + { + if (!$this->EvaluateMultiplicativeExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '+': + case '-'; + if ($this->EvaluateMultiplicativeExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue; + } + else + { + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process a Constant (number of string), retrieve the value of a known variable, or process a function, returning result on the stack. + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateConstantVarOrFunction() + { + if ($this->pos + 1 >= $this->count) + { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + switch ($token[2]) + { + case 'NUMBER': + case 'STRING': + $this->StackPush($token); + return true; + break; + case 'WORD': + case 'SGQA': + if (($this->pos + 1) < $this->count and $this->tokens[($this->pos + 1)][2] == 'LP') + { + return $this->EvaluateFunction(); + } + else + { + if ($this->isValidVariable($token[0])) + { + $this->varsUsed[] = $token[0]; // add this variable to list of those used in this equation + $result = array($this->amVars[$token[0]],$token[1],'NUMBER'); + $this->StackPush($result); + return true; + } + else if ($this->isValidReservedWord($token[0])) + { + $this->reservedWordsUsed[] = $token[0]; + $result = array($this->amReservedWords[$token[0]],$token[1],'NUMBER'); + $this->StackPush($result); + return true; + } + else + { + $this->AddError("Undefined variable or reserved word", $token); + return false; + } + } + break; + case 'COMMA': + --$this->pos; + $this->AddError("Should never get to this line?",$token); + return false; + default: + return false; + break; + } + } + + /* + * Process "a == b", "a eq b", "a != b", "a ne b" + * @return boolean - true if success, false if any error occurred + / + private function EvaluateEqualityExpression() + { + if (!$this->EvaluateRelationExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '==': + case 'eq': + case '!=': + case 'ne': + if ($this->EvaluateRelationExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue; + } + else + { + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process a single expression (e.g. without commas) + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateExpression() + { + if ($this->pos + 2 < $this->count) + { + $token1 = $this->tokens[++$this->pos]; + $token2 = $this->tokens[++$this->pos]; + if ($this->isValidVariable($token1[0]) and $token2[2] == 'ASSIGN') + { + $evalStatus = $this->EvaluateLogicalOrExpression(); + if ($evalStatus) + { + $result = $this->StackPop(); + if (!is_null($result)) + { + $newResult = $token2; + $newResult[2] = 'NUMBER'; + $newResult[0] = $this->setVariableValue($token2[0], $token1[0], $result[0]); + $this->StackPush($newResult); + } + else + { + $evalStatus = false; + } + } + return $evalStatus; + } + else + { + // not an assignment expression, so try something else + $this->pos -= 2; + return $this->EvaluateLogicalOrExpression(); + } + } + else + { + return $this->EvaluateLogicalOrExpression(); + } + } + + /* + * Process "expression [, expression]* + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateExpressions() + { + $evalStatus = $this->EvaluateExpression(); + if (!$evalStatus) + { + return false; + } + + while (++$this->pos < $this->count) { + $token = $this->tokens[$this->pos]; + if ($token[2] == 'RP') + { + return true; // presumbably the end of an expression + } + else if ($token[2] == 'COMMA') + { + if ($this->EvaluateExpression()) + { + $secondResult = $this->StackPop(); + $firstResult = $this->StackPop(); + if (is_null($firstResult)) + { + return false; + } + $this->StackPush($secondResult); + $evalStatus = true; + } + + } + else + { + $this->AddError("Expected expressions separated by commas",$token); + $evalStatus = false; + break; + } + } + while (++$this->pos < $this->count) + { + $token = $this->tokens[$this->pos]; + $this->AddError("Extra token found after Expressions",$token); + $evalStatus = false; + } + return $evalStatus; + } + + /* + * Process a function call + * @return boolean - true if success, false if any error occurred + / + private function EvaluateFunction() + { + $funcNameToken = $this->tokens[$this->pos]; // note that don't need to increment position for functions + $funcName = $funcNameToken[0]; + if (!$this->isValidFunction($funcName)) + { + $this->AddError("Undefined Function", $funcNameToken); + return false; + } + $token2 = $this->tokens[++$this->pos]; + if ($token2[2] != 'LP') + { + $this->AddError("Expected '(' after function name", $token); + } + $params = array(); // will just store array of values, not tokens + while ($this->pos + 1 < $this->count) + { + $token3 = $this->tokens[$this->pos + 1]; + if (count($params) > 0) + { + // should have COMMA or RP + if ($token3[2] == 'COMMA') + { + ++$this->pos; // consume the token so can process next clause + if ($this->EvaluateExpression()) + { + $value = $this->StackPop(); + if (is_null($value)) + { + return false; + } + $params[] = $value[0]; + continue; + } + else + { + $this->AddError("Extra comma found in function", $token3); + return false; + } + } + } + if ($token3[2] == 'RP') + { + ++$this->pos; // consume the token so can process next clause + return $this->RunFunction($funcNameToken,$params); + } + else + { + if ($this->EvaluateExpression()) + { + $value = $this->StackPop(); + if (is_null($value)) + { + return false; + } + $params[] = $value[0]; + continue; + } + else + { + return false; + } + } + } + } + + /* + * Process "a && b" or "a and b" + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateLogicalAndExpression() + { + if (!$this->EvaluateEqualityExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '&&': + case 'and': + if ($this->EvaluateEqualityExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process "a \|\| b" or "a or b" + * @return boolean - true if success, false if any error occurred + / + private function EvaluateLogicalOrExpression() + { + if (!$this->EvaluateLogicalAndExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '\|\|': + case 'or': + if ($this->EvaluateLogicalAndExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + default: + // no more expressions being ORed together, so continue parsing + --$this->pos; + return true; + } + } + // no more tokens to parse + return true; + } + + /* + * Process "a op b" where op in (,/) + @return boolean - true if success, false if any error occurred + / + + private function EvaluateMultiplicativeExpression() + { + if (!$this->EvaluateUnaryExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '': + case '/'; + if ($this->EvaluateUnaryExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /** + * Process expressions including functions and parenthesized blocks + * @return boolean - true if success, false if any error occurred + / + + private function EvaluatePrimaryExpression() + { + if (($this->pos + 1) >= $this->count) { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + if ($token[2] == 'LP') + { + if (!$this->EvaluateExpressions()) + { + return false; + } + $token = $this->tokens[$this->pos]; + if ($token[2] == 'RP') + { + return true; + } + else + { + $this->AddError("Expected ')'", $token); + return false; + } + } + else + { + --$this->pos; + return $this->EvaluateConstantVarOrFunction(); + } + } + + /* + * Process "a op b" where op in (lt, gt, le, ge, <, >, <=, >=) + * @return boolean - true if success, false if any error occurred + / + private function EvaluateRelationExpression() + { + if (!$this->EvaluateAdditiveExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '<': + case 'lt': + case '<='; + case 'le': + case '>': + case 'gt': + case '>='; + case 'ge': + if ($this->EvaluateAdditiveExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process "op a" where op in (+,-,!) + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateUnaryExpression() + { + if (($this->pos + 1) >= $this->count) { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '+': + case '-': + case '!': + if (!$this->EvaluatePrimaryExpression()) + { + return false; + } + return $this->EvalUnary($token); + break; + default: + --$this->pos; + return $this->EvaluatePrimaryExpression(); + } + } + + /* + * Returns array of all reserved words used when parsing a string via sProcessStringContainingExpressions + * @return <type> + / + + public function GetAllReservedWordsUsed() + { + return array_unique($this->allReservedWordsUsed); + } + + /* + * Returns array of all variables used when parsing a string via sProcessStringContainingExpressions + * @return <type> + / + public function GetAllVarsUsed() + { + return array_unique($this->allVarsUsed); + } + + /* + * Return the result of evaluating the equation - NULL if error + * @return mixed + / + public function GetResult() + { + return $this->result[0]; + } + + /* + * Return an array of errors + * @return array + / + public function GetErrors() + { + return $this->errs; + } + + /* + * Return an array of human-readable errors (message, offending token, offset of offending token within equation) + * @return array + / + public function GetReadableErrors() + { + $errs = array(); + foreach ($this->errs as $err) + { + $msg = $err[0]; + $token = $err[1]; + $toshow = 'ERR'; + if (!is_null($token)) + { + $toshow .= '[' . $token[0] . ' @pos=' . $token[1] . ']'; + } + $toshow .= ': ' . $msg; + $errs[] = $toshow; + } + return $errs; + } + + /* + * Return array of list of reserved words used in the equation + * @return <type> + / + + public function GetReservedWordsUsed() + { + return array_unique($this->reservedWordsUsed); + } + + /* + * Return array of the list of variables used in the equation + * @return array + / + public function GetVarsUsed() + { + return array_unique($this->varsUsed); + } + + /* + * Return true if there were syntax or processing errors + * @return boolean + / + public function HasErrors() + { + return (count($this->errs) > 0); + } + + /* + * Return true if there are syntax errors + * @return boolean + / + private function HasSyntaxErrors() + { + // check for bad tokens + // check for unmatched parentheses + // check for undefined variables + // check for undefined functions (but can't easily check allowable # elements?) + + $nesting = 0; + + for ($i=0;$i<$this->count;++$i) + { + $token = $this->tokens[$i]; + switch ($token[2]) + { + case 'LP': + ++$nesting; + break; + case 'RP': + --$nesting; + if ($nesting < 0) + { + $this->AddError("Extra ')' detected", $token); + } + break; + case 'WORD': + case 'SGQA': + if ($i+1 < $this->count and $this->tokens[$i+1][2] == 'LP') + { + if (!$this->isValidFunction($token[0])) + { + $this->AddError("Undefined function", $token); + } + } + else + { + if (!($this->isValidVariable($token[0]) or $this->isValidReservedWord($token[0]))) + { + $this->AddError("Undefined variable or reserved word", $token); + } + } + break; + case 'OTHER': + $this->AddError("Unsupported syntax", $token); + break; + default: + break; + } + } + if ($nesting != 0) + { + $this->AddError("Parentheses not balanced",NULL); + } + return (count($this->errs) > 0); + } + + /* + * Return true if the function name is registered + * @param <type> $name + * @return boolean + / + + private function isValidFunction($name) + { + return array_key_exists($name,$this->amValidFunctions); + } + + /* + * Return true if the reserved word name is registered + * @param <type> $name + * @return boolean + / + private function isValidReservedWord($name) + { + return array_key_exists($name,$this->amReservedWords); + } + + /* + * Return true if the variable name is registered + * @param <type> $name + * @return boolean + / + private function isValidVariable($name) + { + return array_key_exists($name,$this->amVars); + } + + /* + * Process a full string, containing multiple expressions delimited by {}, return a consolidated string + * @param <type> $src + / + + public function sProcessStringContainingExpressions($src) + { + // tokenize string by the {} pattern, properly dealing with strings in quotations, and escaped curly brace values + $stringParts = $this->asSplitStringOnExpressions($src); + + $resolvedParts = array(); + $this->allVarsUsed = array(); + $this->allReservedWordsUsed = array(); + + foreach ($stringParts as $stringPart) + { + if ($stringPart[2] == 'STRING') { + $resolvedParts[] = $stringPart[0]; + } + else { + if ($this->Evaluate(substr($stringPart[0],1,-1))) + { + $resolvedParts[] = $this->GetResult(); + $this->allVarsUsed = array_merge($this->allVarsUsed,$this->GetVarsUsed()); + $this->allReservedWordsUsed = array_merge($this->allReservedWordsUsed, $this->GetReservedWordsUsed()); + } + else + { + // show original and errors in-line? + $resolvedParts[] = '[' . $stringPart[0] . ':' . implode(';',$this->GetReadableErrors()) . ']'; + } + } + } + $result = implode('',$resolvedParts); + return $result; + } + + /* + * Run a registered function + * @param <type> $funcNameToken + * @param <type> $params + * @return boolean + / + private function RunFunction($funcNameToken,$params) + { + $name = $funcNameToken[0]; + if (!$this->isValidFunction($name)) + { + return false; + } + $func = $this->amValidFunctions[$name]; + $funcName = $func[0]; + $numArgs = count($params); + + if (function_exists($funcName)) { + $numArgsAllowed = array_slice($func, 2); + $argsPassed = is_array($params) ? count($params) : 0; + + // for unlimited # parameters + try + { + if (in_array(-1, $numArgsAllowed)) { + $result = $funcName($params); + + // Call function with the params passed + } elseif (in_array($argsPassed, $numArgsAllowed)) { + + switch ($argsPassed) { + case 0: + $result = $funcName(); + break; + case 1: + $result = $funcName($params[0]); + break; + case 2: + $result = $funcName($params[0], $params[1]); + break; + case 3: + $result = $funcName($params[0], $params[1], $params[2]); + break; + case 4: + $result = $funcName($params[0], $params[1], $params[2], $params[3]); + break; + default: + $this->AddError("Error: Unsupported arg count: $funcName(".implode(", ",$params),$funcNameToken); + return false; + } + + } else { + $this->AddError("Error: Incorrect arg count: " . $funcName ."(".implode(", ",$params).")",$funcNameToken); + return false; + } + } + catch (Exception $e) + { + $this->AddError($e->getMessage(),$funcNameToken); + return false; + } + $token = array($result,$funcNameToken[1],'NUMBER'); + $this->StackPush($token); + return true; + } + } + + /* + * Add user functions to array of allowable functions within the equation. + * $functions is an array of key to value mappings like this: + * 'newfunc' => array('my_func_script', 1,3) + * where 'newfunc' is the name of an allowable function wihtin the expression, 'my_func_script' is the registered PHP function name, + * and 1,3 are the list of allowable numbers of paremeters (so my_func() can take 1 or 3 parameters. + * + * @param array $functions + / + + public function RegisterFunctions(array $functions) { + $this->amValidFunctions= array_merge($this->amValidFunctions, $functions); + } + + /* + * Add list of allowable ReservedWord names within the equation + * $varnames is an array of key to value mappings like this: + * 'myvar' => value + * where value is optional (e.g. can be blank), and can be any scalar type (e.g. string, number, but not array) + * the system will use the values as fast lookup when doing calculations, but if it needs to set values, it will call + * the interface function to set the values by name + * + * @param array $varnames + / + public function RegisterReservedWordsUsingMerge(array $varnames) { + $this->amReservedWords = array_merge($this->amReservedWords, $varnames); + } + + public function RegisterReservedWordsUsingReplace(array $varnames) { + $this->amReservedWords = array_merge(array(), $varnames); + } + + /* + * Add list of allowable variable names within the equation + * $varnames is an array of key to value mappings like this: + * 'myvar' => value + * where value is optional (e.g. can be blank), and can be any scalar type (e.g. string, number, but not array) + * the system will use the values as fast lookup when doing calculations, but if it needs to set values, it will call + * the interface function to set the values by name + * + * @param array $varnames + / + public function RegisterVarnamesUsingMerge(array $varnames) { + $this->amVars = array_merge($this->amVars, $varnames); + } + + public function RegisterVarnamesUsingReplace(array $varnames) { + $this->amVars = array_merge(array(), $varnames); + } + + /* + * Set the value of a registered variable + * @param $op - the operator (=,=,/=,+=,-=) + @param <type> $name + * @param <type> $value + / + private function setVariableValue($op,$name,$value) + { + // TODO - set this externally + switch($op) + { + case '=': + $this->amVars[$name] = $value; + break; + case '=': + $this->amVars[$name] = $value; + break; + case '/=': + $this->amVars[$name] /= $value; + break; + case '+=': + $this->amVars[$name] += $value; + break; + case '-=': + $this->amVars[$name] -= $value; + break; + } + return $this->amVars[$name]; + } + + public function asSplitStringOnExpressions($src) + { + // tokenize string by the {} pattern, propertly dealing with strings in quotations, and escaped curly brace values + $tokens0 = preg_split($this->sExpressionRegex,$src,-1,(PREG_SPLIT_NO_EMPTY \| PREG_SPLIT_DELIM_CAPTURE \| PREG_SPLIT_OFFSET_CAPTURE)); + + $tokens = array(); + // Add token_type to $tokens: For each token, test each categorization in order - first match will be the best. + for ($j=0;$j<count($tokens0);++$j) + { + $token = $tokens0[$j]; + if (preg_match($this->sExpressionRegex,$token[0])) + { + $token[2] = 'EXPRESSION'; + } + else + { + $token[2] = 'STRING'; + } + $tokens[] = $token; + } + return $tokens; + } + + /* + * Pop a value token off of the stack + * @return token + / + + private function StackPop() + { + if (count($this->stack) > 0) + { + return array_pop($this->stack); + } + else + { + $this->AddError("Tried to pop value off of empty stack", NULL); + return NULL; + } + } + + /* + * Stack only holds values (number, string), not operators + * @param array $token + / + + private function StackPush(array $token) + { + if ($this->onlyparse) + { + // If only parsing, still want to validate syntax, so use "1" for all variables + switch($token[2]) + { + case 'STRING': + $this->stack[] = array(1,$token[1],'STRING'); + break; + case 'NUMBER': + default: + $this->stack[] = array(1,$token[1],'NUMBER'); + break; + } + } + else + { + $this->stack[] = $token; + } + } + + /* + * Split the source string into tokens, removing whitespace, and categorizing them by type. + * + * @param $src + * @return array + / + + private function amTokenize($src) + { + // $tokens0 = array of tokens from equation, showing value and offset position. Will include SPACE, which should be removed + $tokens0 = preg_split($this->sTokenizerRegex,$src,-1,(PREG_SPLIT_NO_EMPTY \| PREG_SPLIT_DELIM_CAPTURE \| PREG_SPLIT_OFFSET_CAPTURE)); + + // $tokens = array of tokens from equation, showing value, offsete position, and type. Will not contain SPACE, but will contain OTHER + $tokens = array(); + // Add token_type to $tokens: For each token, test each categorization in order - first match will be the best. + for ($j=0;$j<count($tokens0);++$j) + { + for ($i=0;$i<count($this->asCategorizeTokensRegex);++$i) + { + $token = $tokens0[$j][0]; + if (preg_match($this->asCategorizeTokensRegex[$i],$token)) + { + if ($this->asTokenType[$i] !== 'SPACE') { + $tokens0[$j][2] = $this->asTokenType[$i]; + if ($this->asTokenType[$i] == 'STRING') + { + // remove outside quotes + $unquotedToken = stripslashes(substr($token,1,-1)); + $tokens0[$j][0] = $unquotedToken; + } + $tokens[] = $tokens0[$j]; // get first matching non-SPACE token type and push onto $tokens array + } + break; // only get first matching token type + } + } + } + return $tokens; + } + + /* + * Unit test the asSplitStringOnExpressions() function to ensure that accurately parses out all expressions + * surrounded by curly braces, allowing for strings and escaped curly braces. + / + + static function UnitTestStringSplitter() + { + $tests = <<<EOD +"this is a string that contains {something in curly braces)" +This example has escaped curly braces like \{this is not an equation\} +Should the parser check for unmatched { opening curly braces? +What about for unmatched } closing curly braces? +{ANS:name}, you said that you are {ANS:age} years old, and that you have {ANS:numKids} {EVAL:if((numKids==1),'child','children')} and {ANS:numPets} {EVAL:if((numPets==1),'pet','pets')} running around the house. So, you have {EVAL:numKids + numPets} wild {EVAL:if((numKids + numPets ==1),'beast','beasts')} to chase around every day. +Since you have more {EVAL:if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'children','pets')} than you do {EVAL:if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')}, do you feel that the {EVAL:if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')} are at a disadvantage? +EOD; + + $em = new ExpressionManager(); + + foreach(explode("\n",$tests) as $test) + { + $tokens = $em->asSplitStringOnExpressions($test); + print '<b>' . $test . '</b><hr/>'; + print '<code>'; + print implode("<br/>\n",explode("\n",print_r($tokens,TRUE))); + print '</code><hr/>'; + } + } + + /* + * Unit test the Tokenizer - Tokenize and generate a HTML-compatible print-out of a comprehensive set of test cases + / + + static function UnitTestTokenizer() + { + // Comprehensive test cases for tokenizing + $tests = <<<EOD + String: "Can strings contain embedded \"quoted passages\" (and parenthesis + other characters?)?" + String: "can single quoted strings" . 'contain nested \'quoted sections\'?'; + Parens: upcase('hello'); + Numbers: 42 72.35 -15 +37 42A .5 0.7 + And_Or: (this and that or the other); Sandles, sorting; (a && b \|\| c) + Words: hi there, my name is C3PO! + UnaryOps: ++a, --b !b + BinaryOps: (a + b c / d) + Comparators: > >= < <= == != gt ge lt le eq ne (target large gents built agile less equal) + Assign: = += -= = /= + SGQA: 1X6X12 1X6X12ber1 1X6X12ber1_lab1 3583X84X249 + Errors: Apt # 10C; (2 > 0) ? 'hi' : 'there'; array[30]; >>> <<< / this is not a comment / // neither is this +EOD; + + $em = new ExpressionManager(); + + foreach(explode("\n",$tests) as $test) + { + $tokens = $em->amTokenize($test); + print '<b>' . $test . '</b><hr/>'; + print '<code>'; + print implode("<br/>\n",explode("\n",print_r($tokens,TRUE))); + print '</code><hr/>'; + } + } + + /* + * Unit test the Evaluator, allowing for passing in of extra functions, variables, and tests + * @param array $extraFunctions + * @param array $extraVars + * @param <type> $extraTests + / + + static function UnitTestEvaluator(array $extraFunctions=array(), array $extraVars=array(), $extraTests='1~1') + { + // Some test cases for Evaluator + $vars = array( + 'one' =>1, + 'two' =>2, + 'three' =>3, + 'four' =>4, + 'five' =>5, + 'six' =>6, + 'seven' =>7, + 'eight' =>8, + 'nine' =>9, + 'ten' =>10, + 'eleven' => 11, + 'twelve' => 12, + 'half' =>.5, + 'hi' =>'there', + 'hello' =>"Tom", + 'a' =>0, + 'b' =>0, + 'c' =>0, + 'd' =>0, + '12X34X56' =>5, + '12X3X5lab1_ber' =>10, + 'q5pointChoice.code' =>5, + 'q5pointChoice.value' => 'Father', + 'qArrayNumbers.ls1.min.code' => 7, + 'qArrayNumbers.ls1.min.value' => 'I love LimeSurvey', + '12X3X5lab1_ber#2' => 15, + ); + + $reservedWord = array( + 'ADMINEMAIL' =>'{ADMINEMAIL}', + 'ADMINNAME' =>'{ADMINNAME}', + 'AID' =>'{AID}', + 'ANSWERSCLEARED' =>'{ANSWERSCLEARED}', + 'ANSWER' =>'{ANSWER}', + 'ASSESSMENTS' =>'{ASSESSMENTS}', + 'ASSESSMENT_CURRENT_TOTAL' =>'{ASSESSMENT_CURRENT_TOTAL}', + 'ASSESSMENT_HEADING' =>'{ASSESSMENT_HEADING}', + 'CHECKJAVASCRIPT' =>'{CHECKJAVASCRIPT}', + 'CLEARALL' =>'{CLEARALL}', + 'CLOSEWINDOW' =>'{CLOSEWINDOW}', + 'COMPLETED' =>'{COMPLETED}', + 'DATESTAMP' =>'{DATESTAMP}', + 'EMAILCOUNT' =>'{EMAILCOUNT}', + 'EMAIL' =>'{EMAIL}', + 'EXPIRY' =>'{EXPIRY}', + 'FIRSTNAME' =>'{FIRSTNAME}', + 'GID' =>'{GID}', + 'GROUPDESCRIPTION' =>'{GROUPDESCRIPTION}', + 'GROUPNAME' =>'{GROUPNAME}', + 'INSERTANS:123X45X67' =>'{INSERTANS:123X45X67}', + 'INSERTANS:123X45X67ber' =>'{INSERTANS:123X45X67ber}', + 'INSERTANS:123X45X67ber_01a' =>'{INSERTANS:123X45X67ber_01a}', + 'LANGUAGECHANGER' =>'{LANGUAGECHANGER}', + 'LANGUAGE' =>'{LANGUAGE}', + 'LANG' =>'{LANG}', + 'LASTNAME' =>'{LASTNAME}', + 'LOADERROR' =>'{LOADERROR}', + 'LOADFORM' =>'{LOADFORM}', + 'LOADHEADING' =>'{LOADHEADING}', + 'LOADMESSAGE' =>'{LOADMESSAGE}', + 'NAME' =>'{NAME}', + 'NAVIGATOR' =>'{NAVIGATOR}', + 'NOSURVEYID' =>'{NOSURVEYID}', + 'NOTEMPTY' =>'{NOTEMPTY}', + 'NULL' =>'{NULL}', + 'NUMBEROFQUESTIONS' =>'{NUMBEROFQUESTIONS}', + 'OPTOUTURL' =>'{OPTOUTURL}', + 'PASSTHRULABEL' =>'{PASSTHRULABEL}', + 'PASSTHRUVALUE' =>'{PASSTHRUVALUE}', + 'PERCENTCOMPLETE' =>'{PERCENTCOMPLETE}', + 'PERC' =>'{PERC}', + 'PRIVACYMESSAGE' =>'{PRIVACYMESSAGE}', + 'PRIVACY' =>'{PRIVACY}', + 'QID' =>'{QID}', + 'QUESTIONHELPPLAINTEXT' =>'{QUESTIONHELPPLAINTEXT}', + 'QUESTIONHELP' =>'{QUESTIONHELP}', + 'QUESTION_CLASS' =>'{QUESTION_CLASS}', + 'QUESTION_CODE' =>'{QUESTION_CODE}', + 'QUESTION_ESSENTIALS' =>'{QUESTION_ESSENTIALS}', + 'QUESTION_FILE_VALID_MESSAGE' =>'{QUESTION_FILE_VALID_MESSAGE}', + 'QUESTION_HELP' =>'{QUESTION_HELP}', + 'QUESTION_INPUT_ERROR_CLASS' =>'{QUESTION_INPUT_ERROR_CLASS}', + 'QUESTION_MANDATORY' =>'{QUESTION_MANDATORY}', + 'QUESTION_MAN_CLASS' =>'{QUESTION_MAN_CLASS}', + 'QUESTION_MAN_MESSAGE' =>'{QUESTION_MAN_MESSAGE}', + 'QUESTION_NUMBER' =>'{QUESTION_NUMBER}', + 'QUESTION_TEXT' =>'{QUESTION_TEXT}', + 'QUESTION_VALID_MESSAGE' =>'{QUESTION_VALID_MESSAGE}', + 'QUESTION' =>'{QUESTION}', + 'REGISTERERROR' =>'{REGISTERERROR}', + 'REGISTERFORM' =>'{REGISTERFORM}', + 'REGISTERMESSAGE1' =>'{REGISTERMESSAGE1}', + 'REGISTERMESSAGE2' =>'{REGISTERMESSAGE2}', + 'RESTART' =>'{RESTART}', + 'RETURNTOSURVEY' =>'{RETURNTOSURVEY}', + 'SAVEALERT' =>'{SAVEALERT}', + 'SAVEDID' =>'{SAVEDID}', + 'SAVEERROR' =>'{SAVEERROR}', + 'SAVEFORM' =>'{SAVEFORM}', + 'SAVEHEADING' =>'{SAVEHEADING}', + 'SAVEMESSAGE' =>'{SAVEMESSAGE}', + 'SAVE' =>'{SAVE}', + 'SGQ' =>'{SGQ}', + 'SID' =>'{SID}', + 'SITENAME' =>'{SITENAME}', + 'SUBMITBUTTON' =>'{SUBMITBUTTON}', + 'SUBMITCOMPLETE' =>'{SUBMITCOMPLETE}', + 'SUBMITREVIEW' =>'{SUBMITREVIEW}', + 'SURVEYCONTACT' =>'{SURVEYCONTACT}', + 'SURVEYDESCRIPTION' =>'{SURVEYDESCRIPTION}', + 'SURVEYFORMAT' =>'{SURVEYFORMAT}', + 'SURVEYLANGAGE' =>'{SURVEYLANGAGE}', + 'SURVEYLISTHEADING' =>'{SURVEYLISTHEADING}', + 'SURVEYLIST' =>'{SURVEYLIST}', + 'SURVEYNAME' =>'{SURVEYNAME}', + 'SURVEYURL' =>'{SURVEYURL}', + 'TEMPLATECSS' =>'{TEMPLATECSS}', + 'TEMPLATEURL' =>'{TEMPLATEURL}', + 'TEXT' =>'{TEXT}', + 'THEREAREXQUESTIONS' =>'{THEREAREXQUESTIONS}', + 'TIME' =>'{TIME}', + 'TOKEN:EMAIL' =>'{TOKEN:EMAIL}', + 'TOKEN:FIRSTNAME' =>'{TOKEN:FIRSTNAME}', + 'TOKEN:LASTNAME' =>'{TOKEN:LASTNAME}', + 'TOKEN:XXX' =>'{TOKEN:XXX}', + 'TOKENCOUNT' =>'{TOKENCOUNT}', + 'TOKEN_COUNTER' =>'{TOKEN_COUNTER}', + 'TOKEN' =>'{TOKEN}', + 'URL' =>'{URL}', + 'WELCOME' =>'{WELCOME}', + ); + + // Syntax for $tests is~ + // expectedResult~expression + // if the expected result is an error, use NULL for the expected result + $tests = <<<EOD +50~12X34X56 12X3X5lab1_ber +3~a=three +3~c=a +12~c=four +15~c+=a +5~c/=a +-1~c-=six +2~max(one,two) +5~max(one,two,three,four,five) +1024~max(one,(twothree),pow(four,five),six) +1~min(one,two,three,four,five) +27~pow(3,3) +5~hypot(three,four) +0~0 +24~one * two * three * four +-4~five - four - three - two +0~two * three - two - two - two +4~two * three - two +3.1415926535898~pi() +1~pi() == pi() * 2 - pi() +1~sin(pi()/2) +1~sin(0.5 * pi()) +1~sin(pi()/2) == sin(.5 * pi()) +105~5 + 1, 7 * 15 +7~7 +15~10 + 5 +24~12 * 2 +10~13 - 3 +3.5~14 / 4 +5~3 + 1 * 2 +1~one +there~hi +6.25~one * two - three / four + five +1~one + hi +1~two > one +1~two gt one +1~three >= two +1~three ge two +0~four < three +0~four lt three +0~four <= three +0~four le three +0~four == three +0~four eq three +1~four != three +0~four ne four +0~one * hi +5~abs(-five) +0~acos(pi()/2) +0~asin(pi()/2) +10~ceil(9.1) +9~floor(9.9) +32767~getrandmax() +0~rand() +15~sum(one,two,three,four,five) +5~intval(5.7) +1~is_float('5.5') +0~is_float('5') +1~is_numeric(five) +0~is_numeric(hi) +1~is_string(hi) +2.4~(one * two) + (three * four) / (five * six) +1~(one * (two + (three - four) + five) / six) +0~one && 0 +0~two and 0 +1~five && 6 +1~seven && eight +1~one or 0 +1~one \|\| 0 +1~(one and 0) \|\| (two and three) +NULL~hi(there); +NULL~(one * two + (three - four) +NULL~(one * two + (three - four))) +NULL~++a +NULL~--b +11~eleven +144~twelve * twelve +4~if(5 > 7,2,4) +there~if((one > two),'hi','there') +64~if((one < two),pow(2,6),pow(6,2)) +1,2,3,4,5~list(one,two,three,min(four,five,six),max(three,four,five)) +11,12~list(eleven,twelve) +{INSERTANS:123X45X67}~INSERTANS:123X45X67 +{QID}~QID +{ASSESSMENT_HEADING}~ASSESSMENT_HEADING +{TOKEN:FIRSTNAME}~TOKEN:FIRSTNAME +{THEREAREXQUESTIONS}~THEREAREXQUESTIONS +5~q5pointChoice.code +Father~q5pointChoice.value +7~qArrayNumbers.ls1.min.code +I love LimeSurvey~qArrayNumbers.ls1.min.value +15~12X3X5lab1_ber#2 +EOD; + + $em = new ExpressionManager(); + $em->RegisterVarnamesUsingMerge($vars); + $em->RegisterReservedWordsUsingMerge($reservedWord); + + if (is_array($extraVars) and count($extraVars) > 0) + { + $em->RegisterVarnamesUsingMerge($extraVars); + } + if (is_array($extraFunctions) and count($extraFunctions) > 0) + { + $em->RegisterFunctions($extraFunctions); + } + if (is_string($extraTests)) + { + $tests .= "\n" . $extraTests; + } + + print '<table border="1"><tr><th>Expression</th><th>Result</th><th>Expected</th><th>VarsUsed</th><th>ReservedWordsUsed</th><th>Errors</th></tr>'; + foreach(explode("\n",$tests)as $test) + { + $values = explode("~",$test); + $expectedResult = array_shift($values); + $expr = implode("~",$values); + $resultStatus = 'ok'; + print '<tr><td>' . $expr . "</td>\n"; + $status = $em->Evaluate($expr); + $result = $em->GetResult(); + $valToShow = $result; + if (is_null($result)) { + $valToShow = "NULL"; + } + print '<td>' . $valToShow . "</td>\n"; + if ($valToShow != $expectedResult) + { + $resultStatus = 'error'; + } + print "<td class='" . $resultStatus . "'>" . $expectedResult . "</td>\n"; + $varsUsed = $em->GetVarsUsed(); + if (is_array($varsUsed) and count($varsUsed) > 0) { + print '<td>' . implode(', ', $varsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $reservedWordsUsed = $em->GetReservedWordsUsed(); + if (is_array($reservedWordsUsed) and count($reservedWordsUsed) > 0) { + print '<td>' . implode(', ', $reservedWordsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $errs = $em->GetReadableErrors(); + $errStatus = 'ok'; + if (is_array($errs) and count($errs) > 0) { + if ($expectedResult != "NULL") + { + $errStatus = 'error'; // should have been error free + } + print "<td class='" . $errStatus . "'>" . implode("<br/>\n", $errs) . "</td>\n"; + } + else { + if ($expectedResult == "NULL") + { + $errStatus = 'error'; // should have had errors + } + print "<td class='" . $errStatus . "'> </td>\n"; + } + print '</tr>'; + } + print '</table>'; + } + + static function UnitTestProcessStringContainingExpressions() + { + $vars = array( + 'name' => 'Sergei', + 'age' => 45, + 'numKids' => 2, + 'numPets' => 1, + ); + $reservedWords = array( + 'INSERTANS:61764X1X1' => 'Peter', + 'INSERTANS:61764X1X2' => 27, + 'INSERTANS:61764X1X3' => 1, + 'INSERTANS:61764X1X4' => 8 + ); + + $tests = <<<EOD +{name}, you said that you are {age} years old, and that you have {numKids} {if((numKids==1),'child','children')} and {numPets} {if((numPets==1),'pet','pets')} running around the house. So, you have {numKids + numPets} wild {if((numKids + numPets ==1),'beast','beasts')} to chase around every day. +Since you have more {if((numKids > numPets),'children','pets')} than you do {if((numKids > numPets),'pets','children')}, do you feel that the {if((numKids > numPets),'pets','children')} are at a disadvantage? +{INSERTANS:61764X1X1}, you said that you are {INSERTANS:61764X1X2} years old, and that you have {INSERTANS:61764X1X3} {if((INSERTANS:61764X1X3==1),'child','children')} and {INSERTANS:61764X1X4} {if((INSERTANS:61764X1X4==1),'pet','pets')} running around the house. So, you have {INSERTANS:61764X1X3 + INSERTANS:61764X1X4} wild {if((INSERTANS:61764X1X3 + INSERTANS:61764X1X4 ==1),'beast','beasts')} to chase around every day. +Since you have more {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'children','pets')} than you do {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')}, do you feel that the {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')} are at a disadvantage? +EOD; + + $em = new ExpressionManager(); + $em->RegisterVarnamesUsingMerge($vars); + $em->RegisterReservedWordsUsingMerge($reservedWords); + + print '<table border="1"><tr><th>Test</th><th>Result</th><th>VarsUsed</th><th>ReservedWordsUsed</th></tr>'; + foreach(explode("\n",$tests) as $test) + { + print "<tr><td>" . $test . "</td>\n"; + print "<td>" . $em->sProcessStringContainingExpressions($test) . "</td>\n"; + $allVarsUsed = $em->getAllVarsUsed(); + if (is_array($allVarsUsed) and count($allVarsUsed) > 0) { + print "<td>" . implode(', ', $allVarsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $allReservedWordsUsed = $em->getAllReservedWordsUsed(); + if (is_array($allReservedWordsUsed) and count($allReservedWordsUsed) > 0) { + print "<td>" . implode(', ', $allReservedWordsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + print "</tr>\n"; + } + print '</table>'; + } +} + +/* + * Extra Functions can go here. TODO: Find good way to inlcude these extra functions externally. + * Tried via ExpressionManagerFunctions, but they weren't properly included in dFunctionEval.php + / + +function exprmgr_if($test,$ok,$error) +{ + if ($test) + { + return $ok; + } + else + { + return $error; + } +} + +function exprmgr_list($args) +{ + return implode(",",$args); +} + + +?> Index: classes/eval/LimeExpressionManager.php --- classes/eval/LimeExpressionManager.php Locally New +++ classes/eval/LimeExpressionManager.php Locally New @@ -0,0 +1,222 @@ +<?php +/* + * Description of LimeExpressionManager + * + * @author Thomas M. White + / +include_once('ExpressionManager.php'); + +class LimeExpressionManager { + private static $instance; + private $fieldmap; + private $varMap; + private $sgqaMap; + private $tokenMap; + + private $em; // Expression Manager + + // A private constructor; prevents direct creation of object + private function __construct() + { + $this->em = new ExpressionManager(); + } + + // The singleton method + public static function singleton() + { + if (!isset(self::$instance)) { + $c = __CLASS__; + self::$instance = new $c; + } + return self::$instance; + } + + // Prevent users to clone the instance + public function __clone() + { + trigger_error('Clone is not allowed.', E_USER_ERROR); + } + + /* + * Create the arrays needed by ExpressionManager to process LimeSurvey strings. + * Note, the goals is to have this called one time per rendered page (TODO - make that work properly) + * + * @param <type> $sid + * @param <type> $forceRefresh + * @return boolean - true if $fieldmap had been re-created, so ExpressionManager variables need to be re-set + / + + public function setVariableAndTokenMappingsForExpressionManager($sid,$forceRefresh=false) + { + $fieldmap=createFieldMap($sid,$style='full'); + if (!isset($fieldmap)) { + return false; + } + if ($fieldmap === $this->fieldmap and !$forceRefresh) + { + // then this fieldmap is identical to prior one, so don't need to re-create the mappings? + return false; + } + $knownVars = array(); // mapping of VarName to Value + $knownSGQAs = array(); // mapping of SGQA to Value + foreach($fieldmap as $fielddata) + { + $code = $fielddata['fieldname']; + if (preg_match('#^\d+X\d+X#',$code)) + { + switch($fielddata['type']) + { + case '!': //List - dropdown + case '5': //5 POINT CHOICE radio-buttons + case 'D': //DATE + case 'G': //GENDER drop-down list + case 'I': //Language Question + case 'L': //LIST drop-down/radio-button list + case 'N': //NUMERICAL QUESTION TYPE + case 'O': //LIST WITH COMMENT drop-down/radio-button list + textarea + case 'S': //SHORT FREE TEXT + case 'T': //LONG FREE TEXT + case 'U': //HUGE FREE TEXT + case 'X': //BOILERPLATE QUESTION + case 'Y': //YES/NO radio-buttons + case '\|': //File Upload + $varName = $fielddata['title']; + $question = $fielddata['question']; + break; + case '1': //Array (Flexible Labels) dual scale + $varName = $fielddata['title'] . '.' . $fielddata['aid'] . '.' . $fielddata['scale_id']; + $question = $fielddata['question'] . ': ' . $fielddata['subquestion'] . ': ' . $fielddata['scale']; + break; + case 'A': //ARRAY (5 POINT CHOICE) radio-buttons + case 'B': //ARRAY (10 POINT CHOICE) radio-buttons + case 'C': //ARRAY (YES/UNCERTAIN/NO) radio-buttons + case 'E': //ARRAY (Increase/Same/Decrease) radio-buttons + case 'F': //ARRAY (Flexible) - Row Format + case 'H': //ARRAY (Flexible) - Column Format + case 'K': //MULTIPLE NUMERICAL QUESTION + case 'M': //Multiple choice checkbox + case 'P': //Multiple choice with comments checkbox + text + case 'Q': //MULTIPLE SHORT TEXT + case 'R': //RANKING STYLE + $varName = $fielddata['title'] . '.' . $fielddata['aid']; + $question = $fielddata['question'] . ': ' . $fielddata['subquestion']; + break; + case ':': //ARRAY (Multi Flexi) 1 to 10 + case ';': //ARRAY (Multi Flexi) Text + $varName = $fielddata['title'] . '.' . $fielddata['aid']; + $question = $fielddata['question'] . ': ' . $fielddata['subquestion1'] . ': ' . $fielddata['subquestion2']; + break; + } + } + if (isset($_SESSION[$code])) + { + $codeValue = $_SESSION[$code]; + $displayValue= retrieve_Answer($code, $_SESSION['dateformats']['phpdate']); + $knownVars[$varName] = $codeValue; + $knownVars[$varName . '.shown'] = $displayValue; + $knownVars[$varName . '.question']= $question; + $knownSGQAs['INSERTANS:' . $code] = $displayValue; + } + } + $this->varMap = $knownVars; + $this->sgqaMap = $knownSGQAs; + + // Now set tokens + $tokens = array(); // mapping of TOKENS to values - how often does this need to be set? + if (isset($_SESSION['token']) && $_SESSION['token'] != '') + { + //Gather survey data for tokenised surveys, for use in presenting questions + $_SESSION['thistoken']=getTokenData($surveyid, $_SESSION['token']); + } + if (isset($_SESSION['thistoken'])) + { + // TODO - need to explicitly set TOKEN:FIRSTNAME, and related to blank if not using tokens? + foreach (array_keys($_SESSION['thistoken']) as $tokenkey) + { + $tokens["TOKEN:" . strtoupper($tokenkey)] = $_SESSION['thistoken'][$tokenkey]; + } + } + $this->tokenMap = $tokens; + + return true; + } + + /* + * Translate all Expressions, Macros, registered variables, etc. in $string + * @param <type> $string + * @param <type> $sid + * @param boolean $forceRefresh - if true, reset $fieldMap and the derived arrays of registered variables and values + * @return string - the original $string with all replacements done. + / + + static function ProcessString($string,$sid,$forceRefresh=false) + { + $lem = LimeExpressionManager::singleton(); + $em = $lem->em; + if ($lem->setVariableAndTokenMappingsForExpressionManager($sid,$forceRefresh)) + { + // means that some values changed, so need to update what was registered to ExpressionManager + $em->RegisterVarnamesUsingReplace($lem->varMap); + $em->RegisterReservedWordsUsingReplace($lem->sgqaMap); + $em->RegisterReservedWordsUsingMerge($lem->tokenMap); + } + return $em->sProcessStringContainingExpressions(htmlspecialchars_decode($string)); + } + + + /* + * Unit test + / + static function UnitTestProcessStringContainingExpressions() + { + $vars = array( + 'name' => 'Sergei', + 'age' => 45, + 'numKids' => 2, + 'numPets' => 1, + ); + $reservedWords = array( + 'INSERTANS:61764X1X1' => 'Peter', + 'INSERTANS:61764X1X2' => 27, + 'INSERTANS:61764X1X3' => 1, + 'INSERTANS:61764X1X4' => 8 + ); + + $tests = <<<EOD +{name}, you said that you are {age} years old, and that you have {numKids} {if((numKids==1),'child','children')} and {numPets} {if((numPets==1),'pet','pets')} running around the house. So, you have {numKids + numPets} wild {if((numKids + numPets ==1),'beast','beasts')} to chase around every day. +Since you have more {if((numKids > numPets),'children','pets')} than you do {if((numKids > numPets),'pets','children')}, do you feel that the {if((numKids > numPets),'pets','children')} are at a disadvantage? +{INSERTANS:61764X1X1}, you said that you are {INSERTANS:61764X1X2} years old, and that you have {INSERTANS:61764X1X3} {if((INSERTANS:61764X1X3==1),'child','children')} and {INSERTANS:61764X1X4} {if((INSERTANS:61764X1X4==1),'pet','pets')} running around the house. So, you have {INSERTANS:61764X1X3 + INSERTANS:61764X1X4} wild {if((INSERTANS:61764X1X3 + INSERTANS:61764X1X4 ==1),'beast','beasts')} to chase around every day. +Since you have more {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'children','pets')} than you do {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')}, do you feel that the {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')} are at a disadvantage? +EOD; + + $lem = LimeExpressionManager::singleton(); + $em = $lem->em; + + $em->RegisterVarnamesUsingMerge($vars); + $em->RegisterReservedWordsUsingMerge($reservedWords); + + print '<table border="1"><tr><th>Test</th><th>Result</th><th>VarsUsed</th><th>ReservedWordsUsed</th></tr>'; + foreach(explode("\n",$tests) as $test) + { + print "<tr><td>" . $test . "</td>\n"; + print "<td>" . $em->sProcessStringContainingExpressions($test) . "</td>\n"; + $allVarsUsed = $em->getAllVarsUsed(); + if (is_array($allVarsUsed) and count($allVarsUsed) > 0) { + print "<td>" . implode(', ', $allVarsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $allReservedWordsUsed = $em->getAllReservedWordsUsed(); + if (is_array($allReservedWordsUsed) and count($allReservedWordsUsed) > 0) { + print "<td>" . implode(', ', $allReservedWordsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + print "</tr>\n"; + } + print '</table>'; + } +} +?> Index: classes/eval/Test_ExpressionManager_Evaluate.php --- classes/eval/Test_ExpressionManager_Evaluate.php Locally New +++ classes/eval/Test_ExpressionManager_Evaluate.php Locally New @@ -0,0 +1,29 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <style type="text/css"> + <!-- +.error { + background-color: #ff0000; +} +.ok { + background-color: #00ff00 +} + --> + </style> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; +// include 'ExpressionManagerFunctions.php'; +// ExpressionManager::UnitTestEvaluator($exprmgr_functions,$exprmgr_extraVars,$exprmgr_extraTests); + ExpressionManager::UnitTestEvaluator(); + ?> + </body> +</html> Index: classes/eval/Test_ExpressionManager_ProcessStringContainingExpressions.php --- classes/eval/Test_ExpressionManager_ProcessStringContainingExpressions.php Locally New +++ classes/eval/Test_ExpressionManager_ProcessStringContainingExpressions.php Locally New @@ -0,0 +1,17 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <title></title> + </head> + <body> + <?php + include 'LimeExpressionManager.php'; + LimeExpressionManager::UnitTestProcessStringContainingExpressions(); + ?> + </body> +</html> Index: classes/eval/Test_ExpressionManager_Tokenizer.php --- classes/eval/Test_ExpressionManager_Tokenizer.php Locally New +++ classes/eval/Test_ExpressionManager_Tokenizer.php Locally New @@ -0,0 +1,18 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; + ExpressionManager::UnitTestTokenizer(); + ExpressionManager::UnitTestStringSplitter(); + ?> + </body> +</html> Index: common_functions.php --- common_functions.php Base (BASE) +++ common_functions.php Locally Modified (Based On LOCAL) @@ -15,6 +15,7 @@ / +include_once('/classes/eval/LimeExpressionManager.php'); /** * This function gives back an array that defines which survey permissions and what part of the CRUD+Import+Export subpermissions is available. @@ -3147,7 +3148,7 @@ if (strpos($line, "{QUESTIONHELPPLAINTEXT}") !== false) $line=str_replace("{QUESTIONHELPPLAINTEXT}", strip_tags(addslashes($help)), $line); } - $line=insertansReplace($line); +// $line=insertansReplace($line); if (strpos($line, "{SUBMITCOMPLETE}") !== false) $line=str_replace("{SUBMITCOMPLETE}", "<strong>".$clang->gT("Thank you!")."<br /><br />".$clang->gT("You have completed answering the questions in this survey.")."</strong><br /><br />".$clang->gT("Click on 'Submit' now to complete the process and save your answers."), $line); if (strpos($line, "{SUBMITREVIEW}") !== false) { @@ -3160,8 +3161,10 @@ $line=str_replace("{SUBMITREVIEW}", $strreview, $line); } - $line=tokenReplace($line); +// $line=tokenReplace($line); + $line = LimeExpressionManager::ProcessString($line, $surveyid); + if (strpos($line, "{ANSWERSCLEARED}") !== false) $line=str_replace("{ANSWERSCLEARED}", $clang->gT("Answers Cleared"), $line); if (strpos($line, "{RESTART}") !== false) { issue_05103-20110616.patch (90,972 bytes)

TMSWhite 2011-06-16 08:58 reporter	ExpressionManager-plus-all-question-types.lss (169,303 bytes)

TMSWhite 2011-06-16 09:03 reporter ~15451 Last edited: 2011-06-16 09:46 View 2 revisions	First tests of integrating ExpressionManager into the LS1 branch have been successful. At present, it is only integrated into template_replace, but it should not be hard to put it into the other places that do {} replacement. Please give this a try. (1) Apply patch issue_05103-20110616a.patch (2) Try survey ExpressionManager-plus-all-question-types.lss (a) 1st 2 pages show conditional substitution of 4 variables to change wording of question (b) 3rd page tests all question types, and reports (on 4th page), table of question asked, answer given, and code value, all using named variables (rather than SGQA codes)

TMSWhite 2011-06-16 09:45 reporter	issue_05103-20110616a.patch (90,279 bytes) # This patch file was generated by NetBeans IDE # Following Index: paths are relative to: C:\xampp\htdocs\limesurvey # This patch can be applied using context Tools: Patch action on respective folder. # It uses platform neutral UTF-8 encoding and \n newlines. # Above lines and this line are ignored by the patching process. Index: classes/dTexts/dTexts.php --- classes/dTexts/dTexts.php Base (BASE) +++ classes/dTexts/dTexts.php Locally Modified (Based On LOCAL) @@ -18,6 +18,7 @@ { $data=explode(':',$str); $funcName=array_shift($data); + $data = array(implode(':',$data)); try { $func = dTexts::loadFunction($funcName); @@ -39,7 +40,7 @@ / public static function loadFunction($name) { - $name=ucfirst(strtolower($name)); + $name=ucwords($name); $fileName='./classes/dTexts/dFunctions/dFunction'.$name.'.php'; if(!file_exists($fileName)) { Index: classes/eval/ExpressionManager.php --- classes/eval/ExpressionManager.php Locally New +++ classes/eval/ExpressionManager.php Locally New @@ -0,0 +1,1865 @@ +<?php +/* + * Description of ExpressionManager + * (1) Does safe evaluation of PHP expressions. Only registered Functions, Variables, and ReservedWords are allowed. + * (a) Functions include any math, string processing, conditional, formatting, etc. functions + * (b) Variables are typically the question name (question.title) + * (c) ReservedWords are any LimeReplacementField or Token, including all INSERTANS:SGQA codes + * (2) This class can replace LimeSurvey's current process of resolving strings that contain LimeReplacementFields + * (a) String is split by expressions (by curly braces, but safely supporting strings and escaped curly braces) + * (b) Expressions (things surrounded by curly braces) are evaluated - thereby doing LimeReplacementField substitution and/or more complex calculations + * (c) Non-expressions are left intact + * (d) The array of stringParts are re-joined to create the desired final string. + * + * At present, all variables are read-only, but this could be extended to support creation of temporary variables and/or read-write access to registered variables + * + * @author Thomas M. White + / + +class ExpressionManager { + // These three variables are effectively static once constructed + private $sExpressionRegex; + private $asTokenType; + private $sTokenizerRegex; + private $asCategorizeTokensRegex; + private $amValidFunctions; // names and # params of valid functions + private $amVars; // names and values of valid variables + private $amReservedWords; // names and values of valid reserved words + + // Thes variables are used while processing the equation + private $expr; // the source expression + private $tokens; // the list of generated tokens + private $count; // total number of $tokens + private $pos; // position within the $token array while processing equation + private $errs; // array of syntax errors + private $onlyparse; + private $stack; // stack of intermediate results + private $result; // final result of evaluating the expression; + private $evalStatus; // true if $result is a valid result, and there are no serious errors + private $varsUsed; // list of variables referenced in the equation + private $reservedWordsUsed; // list of reserved words used in the equation + + // These variables are only used by sProcessStringContainingExpressions + private $allVarsUsed; // full list of variables used within the string, even if contains multiple expressions + private $allReservedWordsUsed; // full list of reserved words used in the string, even if contains multiple expresions + + function __construct() + { + // List of token-matching regular expressions + $regex_dq_string = '(?<!\\\\)".?(?<!\\\\)"'; + $regex_sq_string = '(?<!\\\\)\'.?(?<!\\\\)\''; + $regex_whitespace = '\s+'; + $regex_lparen = '$'; + $regex_rparen = '$'; + $regex_comma = ','; + $regex_not = '!'; + $regex_inc_dec = '\+\+\|--'; + $regex_binary = '[+/-]'; + $regex_compare = '<=\|<\|>=\|>\|==\|!=\|\ble\b\|\blt\b\|\bge\b\|\bgt\b\|\beq\b\|\bne\b'; + $regex_assign = '=\|\+=\|-=\|\=\|/='; + $regex_sgqa = '[0-9]+X[0-9]+X[0-9]+[A-Z0-9_]'; + $regex_word = '[A-Z][A-Z0-9_]:?[A-Z0-9_]'; + $regex_number = '[0-9]+\.?[0-9]\|\.[0-9]+'; + $regex_andor = '\band\b\|\bor\b\|&&\|\\|\\|'; + + $this->sExpressionRegex = '#((?<!\\\\){(' . $regex_dq_string . '\|' . $regex_sq_string . '\|.?)(?<!\\\\)})#'; + + // asTokenRegex and t_tokey_type must be kept in sync (same number and order) + $asTokenRegex = array( + $regex_dq_string, + $regex_sq_string, + $regex_whitespace, + $regex_lparen, + $regex_rparen, + $regex_comma, + $regex_andor, + $regex_compare, + $regex_sgqa, + $regex_word, + $regex_number, + $regex_not, + $regex_inc_dec, + $regex_assign, + $regex_binary, + ); + + $this->asTokenType = array( + 'STRING', + 'STRING', + 'SPACE', + 'LP', + 'RP', + 'COMMA', + 'AND_OR', + 'COMPARE', + 'SGQA', + 'WORD', + 'NUMBER', + 'NOT', + 'OTHER', + 'ASSIGN', + 'BINARYOP', + ); + + // $sTokenizerRegex - a single regex used to split and equation into tokens + $this->sTokenizerRegex = '#(' . implode('\|',$asTokenRegex) . ')#i'; + + // $asCategorizeTokensRegex - an array of patterns so can categorize the type of token found - would be nice if could get this from preg_split + // Adding ability to capture 'OTHER' type, which indicates an error - unsupported syntax element + $this->asCategorizeTokensRegex = preg_replace("#^(.)$#","#^$1$#i",$asTokenRegex); + $this->asCategorizeTokensRegex[] = '/.+/'; + $this->asTokenType[] = 'OTHER'; + + // Each allowed function is a mapping from local name to external name + number of arguments + // Functions can have -1 (meaning unlimited), or a list of serveral allowable #s of arguments. + $this->amValidFunctions = array( + 'abs' =>array('abs','Absolute value',1), + 'acos' =>array('acos','Arc cosine',1), + 'acosh' =>array('acosh','Inverse hyperbolic cosine',1), + 'asin' =>array('asin','Arc sine',1), + 'asinh' =>array('asinh','Inverse hyperbolic sine',1), + 'atan2' =>array('atan2','Arc tangent of two variables',2), + 'atan' =>array('atan','Arc tangent',1), + 'atanh' =>array('atanh','Inverse hyperbolic tangent',1), + 'base_convert' =>array('base_convert','Convert a number between arbitrary bases',3), + 'bindec' =>array('bindec','Binary to decimal',1), + 'ceil' =>array('ceil','Round fractions up',1), + 'cos' =>array('cos','Cosine',1), + 'cosh' =>array('cosh','Hyperbolic cosine',1), + 'decbin' =>array('decbin','Decimal to binary',1), + 'dechex' =>array('dechex','Decimal to hexadecimal',1), + 'decoct' =>array('decoct','Decimal to octal',1), + 'deg2rad' =>array('deg2rad','Converts the number in degrees to the radian equivalent',1), + 'exp' =>array('exp','Calculates the exponent of e',1), + 'expm1' =>array('expm1','Returns exp(number) - 1, computed in a way that is accurate even when the value of number is close to zero',1), + 'floor' =>array('floor','Round fractions down',1), + 'fmod' =>array('fmod','Returns the floating point remainder (modulo) of the division of the arguments',2), + 'getrandmax' =>array('getrandmax','Show largest possible random value',0), + 'hexdec' =>array('hexdec','Hexadecimal to decimal',1), + 'hypot' =>array('hypot','Calculate the length of the hypotenuse of a right-angle triangle',2), + 'is_finite' =>array('is_finite','Finds whether a value is a legal finite number',1), + 'is_infinite' =>array('is_infinite','Finds whether a value is infinite',1), + 'is_nan' =>array('is_nan','Finds whether a value is not a number',1), + 'lcg_value' =>array('lcg_value','Combined linear congruential generator',0), + 'log10' =>array('log10','Base-10 logarithm',1), + 'log1p' =>array('log1p','Returns log(1 + number), computed in a way that is accurate even when the value of number is close to zero',1), + 'log' =>array('log','Natural logarithm',1,2), + 'max' =>array('max','Find highest value',-1), + 'min' =>array('min','Find lowest value',-1), + 'mt_getrandmax' =>array('mt_getrandmax','Show largest possible random value',0), + 'mt_rand' =>array('mt_rand','Generate a better random value',0,2), + 'mt_srand' =>array('mt_srand','Seed the better random number generator',0,1), + 'octdec' =>array('octdec','Octal to decimal',1), + 'pi' =>array('pi','Get value of pi',0), + 'pow' =>array('pow','Exponential expression',2), + 'rad2deg' =>array('rad2deg','Converts the radian number to the equivalent number in degrees',1), + 'rand' =>array('rand','Generate a random integer',0,2), + 'round' =>array('round','Rounds a float',1,2,3), + 'sin' =>array('sin','Sine',1), + 'sinh' =>array('sinh','Hyperbolic sine',1), + 'sqrt' =>array('sqrt','Square root',1), + 'srand' =>array('srand','Seed the random number generator',0,1), + 'sum' =>array('array_sum','Calculate the sum of values in an array',-1), + 'tan' =>array('tan','Tangent',1), + 'tanh' =>array('tanh','Hyperbolic tangent',1), + + 'empty' =>array('empty','Determine whether a variable is empty',1), + 'intval' =>array('intval','Get the integer value of a variable',1,2), + 'is_bool' =>array('is_bool','Finds out whether a variable is a boolean',1), + 'is_float' =>array('is_float','Finds whether the type of a variable is float',1), + 'is_int' =>array('is_int','Find whether the type of a variable is integer',1), + 'is_null' =>array('is_null','Finds whether a variable is NULL',1), + 'is_numeric' =>array('is_numeric','Finds whether a variable is a number or a numeric string',1), + 'is_scalar' =>array('is_scalar','Finds whether a variable is a scalar',1), + 'is_string' =>array('is_string','Find whether the type of a variable is string',1), + + 'addcslashes' =>array('addcslashes','Quote string with slashes in a C style',2), + 'addslashes' =>array('addslashes','Quote string with slashes',1), + 'bin2hex' =>array('bin2hex','Convert binary data into hexadecimal representation',1), + 'chr' =>array('chr','Return a specific character',1), + 'chunk_split' =>array('chunk_split','Split a string into smaller chunks',1,2,3), + 'convert_uudecode' =>array('convert_uudecode','Decode a uuencoded string',1), + 'convert_uuencode' =>array('convert_uuencode','Uuencode a string',1), + 'count_chars' =>array('count_chars','Return information about characters used in a string',1,2), + 'crc32' =>array('crc32','Calculates the crc32 polynomial of a string',1), + 'crypt' =>array('crypt','One-way string hashing',1,2), + 'hebrev' =>array('hebrev','Convert logical Hebrew text to visual text',1,2), + 'hebrevc' =>array('hebrevc','Convert logical Hebrew text to visual text with newline conversion',1,2), + 'html_entity_decode' =>array('html_entity_decode','Convert all HTML entities to their applicable characters',1,2,3), + 'htmlentities' =>array('htmlentities','Convert all applicable characters to HTML entities',1,2,3), + 'htmlspecialchars_decode' =>array('htmlspecialchars_decode','Convert special HTML entities back to characters',1,2), + 'htmlspecialchars' =>array('htmlspecialchars','Convert special characters to HTML entities',1,2,3,4), + 'implode' =>array('implode','Join array elements with a string',-1), + 'lcfirst' =>array('lcfirst','Make a string\'s first character lowercase',1), + 'levenshtein' =>array('levenshtein','Calculate Levenshtein distance between two strings',2,5), + 'ltrim' =>array('ltrim','Strip whitespace (or other characters) from the beginning of a string',1,2), + 'md5' =>array('md5','Calculate the md5 hash of a string',1), + 'metaphone' =>array('metaphone','Calculate the metaphone key of a string',1,2), + 'money_format' =>array('money_format','Formats a number as a currency string',1,2), + 'nl2br' =>array('nl2br','Inserts HTML line breaks before all newlines in a string',1,2), + 'number_format' =>array('number_format','Format a number with grouped thousands',1,2,4), + 'ord' =>array('ord','Return ASCII value of character',1), + 'quoted_printable_decode' =>array('quoted_printable_decode','Convert a quoted-printable string to an 8 bit string',1), + 'quoted_printable_encode' =>array('quoted_printable_encode','Convert a 8 bit string to a quoted-printable string',1), + 'quotemeta' =>array('quotemeta','Quote meta characters',1), + 'rtrim' =>array('rtrim','Strip whitespace (or other characters) from the end of a string',1,2), + 'sha1' =>array('sha1','Calculate the sha1 hash of a string',1), + 'similar_text' =>array('similar_text','Calculate the similarity between two strings',1,2), + 'soundex' =>array('soundex','Calculate the soundex key of a string',1), + 'sprintf' =>array('sprintf','Return a formatted string',-1), + 'str_ireplace' =>array('str_ireplace','Case-insensitive version of str_replace',3), + 'str_pad' =>array('str_pad','Pad a string to a certain length with another string',2,3,4), + 'str_repeat' =>array('str_repeat','Repeat a string',2), + 'str_replace' =>array('str_replace','Replace all occurrences of the search string with the replacement string',3), + 'str_rot13' =>array('str_rot13','Perform the rot13 transform on a string',1), + 'str_shuffle' =>array('str_shuffle','Randomly shuffles a string',1), + 'str_word_count' =>array('str_word_count','Return information about words used in a string',1), + 'strcasecmp' =>array('strcasecmp','Binary safe case-insensitive string comparison',2), + 'strcmp' =>array('strcmp','Binary safe string comparison',2), + 'strcoll' =>array('strcoll','Locale based string comparison',2), + 'strcspn' =>array('strcspn','Find length of initial segment not matching mask',2,3,4), + 'strip_tags' =>array('strip_tags','Strip HTML and PHP tags from a string',1,2), + 'stripcslashes' =>array('stripcslashes','Un-quote string quoted with addcslashes',1), + 'stripos' =>array('stripos','Find position of first occurrence of a case-insensitive string',2,3), + 'stripslashes' =>array('stripslashes','Un-quotes a quoted string',1), + 'stristr' =>array('stristr','Case-insensitive strstr',2,3), + 'strlen' =>array('strlen','Get string length',1), + 'strnatcasecmp' =>array('strnatcasecmp','Case insensitive string comparisons using a "natural order" algorithm',2), + 'strnatcmp' =>array('strnatcmp','String comparisons using a "natural order" algorithm',2), + 'strncasecmp' =>array('strncasecmp','Binary safe case-insensitive string comparison of the first n characters',3), + 'strncmp' =>array('strncmp','Binary safe string comparison of the first n characters',3), + 'strpbrk' =>array('strpbrk','Search a string for any of a set of characters',2), + 'strpos' =>array('strpos','Find position of first occurrence of a string',2,3), + 'strrchr' =>array('strrchr','Find the last occurrence of a character in a string',2), + 'strrev' =>array('strrev','Reverse a string',1), + 'strripos' =>array('strripos','Find position of last occurrence of a case-insensitive string in a string',2,3), + 'strrpos' =>array('strrpos','Find the position of the last occurrence of a substring in a string',2,3), + 'strspn' =>array('Finds the length of the initial segment of a string consisting entirely of characters contained within a given mask.',2,3,4), + 'strstr' =>array('strstr','Find first occurrence of a string',2,3), + 'strtolower' =>array('strtolower','Make a string lowercase',1), + 'strtoupper' =>array('strtoupper','Make a string uppercase',1), + 'strtr' =>array('strtr','Translate characters or replace substrings',3), + 'substr_compare'=>array('substr_compare','Binary safe comparison of two strings from an offset, up to length characters',3,4,5), + 'substr_count' =>array('substr_count','Count the number of substring occurrences',2,3,4), + 'substr_replace'=>array('substr_replace','Replace text within a portion of a string',3,4), + 'substr' =>array('substr','Return part of a string',2,3), + 'ucfirst' =>array('ucfirst','Make a string\'s first character uppercase',1), + 'ucwords' =>array('ucwords','Uppercase the first character of each word in a string',1), + + 'stddev' =>array('stats_standard_deviation','Returns the standard deviation',-1), + + // Locally declared functions + 'if' => array('exprmgr_if','Excel-style if(test,result_if_true,result_if_false)',3), + 'list' => array('exprmgr_list','Return comma-separated list of values',-1), + ); + + $this->amVars = array(); + $this->amReservedWords = array(); + + } + + /** + * Add an error to the error log + * + * @param <type> $errMsg + * @param <type> $token + / + private function AddError($errMsg, $token) + { + $this->errs[] = array($errMsg, $token); + } + + /* + * EvalBinary() computes binary expressions, such as (a or b), (c * d), popping the top two entries off the + * stack and pushing the result back onto the stack. + * + * @param array $token + * @return boolean - false if there is any error, else true + / + + private function EvalBinary(array $token) + { + if (count($this->stack) < 2) + { + $this->AddError("Unable to evaluate binary operator - fewer than 2 entries on stack", $token); + return false; + } + $arg2 = $this->StackPop(); + $arg1 = $this->StackPop(); + if (is_null($arg1) or is_null($arg2)) + { + $this->AddError("Invalid value(s) on the stack", $token); + return false; + } + // TODO: try to determine datatype? + switch(strtolower($token[0])) + { + case 'or': + case '\|\|': + $result = array(($arg1[0] or $arg2[0]),$token[1],'NUMBER'); + break; + case 'and': + case '&&': + $result = array(($arg1[0] and $arg2[0]),$token[1],'NUMBER'); + break; + case '==': + case 'eq': + $result = array(($arg1[0] == $arg2[0]),$token[1],'NUMBER'); + break; + case '!=': + case 'ne': + $result = array(($arg1[0] != $arg2[0]),$token[1],'NUMBER'); + break; + case '<': + case 'lt': + $result = array(($arg1[0] < $arg2[0]),$token[1],'NUMBER'); + break; + case '<='; + case 'le': + $result = array(($arg1[0] <= $arg2[0]),$token[1],'NUMBER'); + break; + case '>': + case 'gt': + $result = array(($arg1[0] > $arg2[0]),$token[1],'NUMBER'); + break; + case '>='; + case 'ge': + $result = array(($arg1[0] >= $arg2[0]),$token[1],'NUMBER'); + break; + case '+': + $result = array(($arg1[0] + $arg2[0]),$token[1],'NUMBER'); + break; + case '-': + $result = array(($arg1[0] - $arg2[0]),$token[1],'NUMBER'); + break; + case '': + $result = array(($arg1[0] * $arg2[0]),$token[1],'NUMBER'); + break; + case '/'; + $result = array(($arg1[0] / $arg2[0]),$token[1],'NUMBER'); + break; + } + $this->StackPush($result); + return true; + } + + /** + * Processes operations like +a, -b, !c + * @param array $token + * @return boolean - true if success, false if any error occurred + / + + private function EvalUnary(array $token) + { + if (count($this->stack) < 1) + { + $this->AddError("Unable to evaluate unary operator - no entries on stack", $token); + return false; + } + $arg1 = $this->StackPop(); + if (is_null($arg1)) + { + $this->AddError("Invalid value(s) on the stack", $token); + return false; + } + // TODO: try to determine datatype? + switch($token[0]) + { + case '+': + $result = array((+$arg1[0]),$token[1],'NUMBER'); + break; + case '-': + $result = array((-$arg1[0]),$token[1],'NUMBER'); + break; + case '!'; + $result = array((!$arg[0]),$token[1],'NUMBER'); + break; + } + $this->StackPush($result); + return true; + } + + + /* + * Main entry function + * @param <type> $expr + * @param <type> $onlyparse - if true, then validate the syntax without computing an answer + * @return boolean - true if success, false if any error occurred + / + + public function Evaluate($expr, $onlyparse=false) + { + $this->expr = $expr; + $this->tokens = $this->amTokenize($expr); + $this->count = count($this->tokens); + $this->pos = -1; // starting position within array (first act will be to increment it) + $this->errs = array(); + $this->onlyparse = $onlyparse; + $this->stack = array(); + $this->evalStatus = false; + $this->result = NULL; + $this->varsUsed = array(); + $this->reservedWordsUsed = array(); + + if ($this->HasSyntaxErrors()) { + return false; + } + else if ($this->EvaluateExpressions()) + { + if ($this->pos < $this->count) + { + $this->AddError("Extra tokens found starting at", $this->tokens[$this->pos]); + return false; + } + $this->result = $this->StackPop(); + if (is_null($this->result)) + { + return false; + } + if (count($this->stack) == 0) + { + $this->evalStatus = true; + return true; + } + else + { + $this-AddError("Unbalanced equation - values left on stack",NULL); + return false; + } + } + else + { + $this->AddError("Not a valid expression",NULL); + return false; + } + } + + + /* + * Process "a op b" where op in (+,-,concatenate) + * @return boolean - true if success, false if any error occurred + / + private function EvaluateAdditiveExpression() + { + if (!$this->EvaluateMultiplicativeExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '+': + case '-'; + if ($this->EvaluateMultiplicativeExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue; + } + else + { + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process a Constant (number of string), retrieve the value of a known variable, or process a function, returning result on the stack. + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateConstantVarOrFunction() + { + if ($this->pos + 1 >= $this->count) + { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + switch ($token[2]) + { + case 'NUMBER': + case 'STRING': + $this->StackPush($token); + return true; + break; + case 'WORD': + case 'SGQA': + if (($this->pos + 1) < $this->count and $this->tokens[($this->pos + 1)][2] == 'LP') + { + return $this->EvaluateFunction(); + } + else + { + if ($this->isValidVariable($token[0])) + { + $this->varsUsed[] = $token[0]; // add this variable to list of those used in this equation + $result = array($this->amVars[$token[0]],$token[1],'NUMBER'); + $this->StackPush($result); + return true; + } + else if ($this->isValidReservedWord($token[0])) + { + $this->reservedWordsUsed[] = $token[0]; + $result = array($this->amReservedWords[$token[0]],$token[1],'NUMBER'); + $this->StackPush($result); + return true; + } + else + { + $this->AddError("Undefined variable or reserved word", $token); + return false; + } + } + break; + case 'COMMA': + --$this->pos; + $this->AddError("Should never get to this line?",$token); + return false; + default: + return false; + break; + } + } + + /* + * Process "a == b", "a eq b", "a != b", "a ne b" + * @return boolean - true if success, false if any error occurred + / + private function EvaluateEqualityExpression() + { + if (!$this->EvaluateRelationExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '==': + case 'eq': + case '!=': + case 'ne': + if ($this->EvaluateRelationExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue; + } + else + { + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process a single expression (e.g. without commas) + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateExpression() + { + if ($this->pos + 2 < $this->count) + { + $token1 = $this->tokens[++$this->pos]; + $token2 = $this->tokens[++$this->pos]; + if ($this->isValidVariable($token1[0]) and $token2[2] == 'ASSIGN') + { + $evalStatus = $this->EvaluateLogicalOrExpression(); + if ($evalStatus) + { + $result = $this->StackPop(); + if (!is_null($result)) + { + $newResult = $token2; + $newResult[2] = 'NUMBER'; + $newResult[0] = $this->setVariableValue($token2[0], $token1[0], $result[0]); + $this->StackPush($newResult); + } + else + { + $evalStatus = false; + } + } + return $evalStatus; + } + else + { + // not an assignment expression, so try something else + $this->pos -= 2; + return $this->EvaluateLogicalOrExpression(); + } + } + else + { + return $this->EvaluateLogicalOrExpression(); + } + } + + /* + * Process "expression [, expression]* + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateExpressions() + { + $evalStatus = $this->EvaluateExpression(); + if (!$evalStatus) + { + return false; + } + + while (++$this->pos < $this->count) { + $token = $this->tokens[$this->pos]; + if ($token[2] == 'RP') + { + return true; // presumbably the end of an expression + } + else if ($token[2] == 'COMMA') + { + if ($this->EvaluateExpression()) + { + $secondResult = $this->StackPop(); + $firstResult = $this->StackPop(); + if (is_null($firstResult)) + { + return false; + } + $this->StackPush($secondResult); + $evalStatus = true; + } + + } + else + { + $this->AddError("Expected expressions separated by commas",$token); + $evalStatus = false; + break; + } + } + while (++$this->pos < $this->count) + { + $token = $this->tokens[$this->pos]; + $this->AddError("Extra token found after Expressions",$token); + $evalStatus = false; + } + return $evalStatus; + } + + /* + * Process a function call + * @return boolean - true if success, false if any error occurred + / + private function EvaluateFunction() + { + $funcNameToken = $this->tokens[$this->pos]; // note that don't need to increment position for functions + $funcName = $funcNameToken[0]; + if (!$this->isValidFunction($funcName)) + { + $this->AddError("Undefined Function", $funcNameToken); + return false; + } + $token2 = $this->tokens[++$this->pos]; + if ($token2[2] != 'LP') + { + $this->AddError("Expected '(' after function name", $token); + } + $params = array(); // will just store array of values, not tokens + while ($this->pos + 1 < $this->count) + { + $token3 = $this->tokens[$this->pos + 1]; + if (count($params) > 0) + { + // should have COMMA or RP + if ($token3[2] == 'COMMA') + { + ++$this->pos; // consume the token so can process next clause + if ($this->EvaluateExpression()) + { + $value = $this->StackPop(); + if (is_null($value)) + { + return false; + } + $params[] = $value[0]; + continue; + } + else + { + $this->AddError("Extra comma found in function", $token3); + return false; + } + } + } + if ($token3[2] == 'RP') + { + ++$this->pos; // consume the token so can process next clause + return $this->RunFunction($funcNameToken,$params); + } + else + { + if ($this->EvaluateExpression()) + { + $value = $this->StackPop(); + if (is_null($value)) + { + return false; + } + $params[] = $value[0]; + continue; + } + else + { + return false; + } + } + } + } + + /* + * Process "a && b" or "a and b" + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateLogicalAndExpression() + { + if (!$this->EvaluateEqualityExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '&&': + case 'and': + if ($this->EvaluateEqualityExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process "a \|\| b" or "a or b" + * @return boolean - true if success, false if any error occurred + / + private function EvaluateLogicalOrExpression() + { + if (!$this->EvaluateLogicalAndExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '\|\|': + case 'or': + if ($this->EvaluateLogicalAndExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + default: + // no more expressions being ORed together, so continue parsing + --$this->pos; + return true; + } + } + // no more tokens to parse + return true; + } + + /* + * Process "a op b" where op in (,/) + @return boolean - true if success, false if any error occurred + / + + private function EvaluateMultiplicativeExpression() + { + if (!$this->EvaluateUnaryExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '': + case '/'; + if ($this->EvaluateUnaryExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /** + * Process expressions including functions and parenthesized blocks + * @return boolean - true if success, false if any error occurred + / + + private function EvaluatePrimaryExpression() + { + if (($this->pos + 1) >= $this->count) { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + if ($token[2] == 'LP') + { + if (!$this->EvaluateExpressions()) + { + return false; + } + $token = $this->tokens[$this->pos]; + if ($token[2] == 'RP') + { + return true; + } + else + { + $this->AddError("Expected ')'", $token); + return false; + } + } + else + { + --$this->pos; + return $this->EvaluateConstantVarOrFunction(); + } + } + + /* + * Process "a op b" where op in (lt, gt, le, ge, <, >, <=, >=) + * @return boolean - true if success, false if any error occurred + / + private function EvaluateRelationExpression() + { + if (!$this->EvaluateAdditiveExpression()) + { + return false; + } + while (($this->pos + 1) < $this->count) + { + $token = $this->tokens[++$this->pos]; + switch (strtolower($token[0])) + { + case '<': + case 'lt': + case '<='; + case 'le': + case '>': + case 'gt': + case '>='; + case 'ge': + if ($this->EvaluateAdditiveExpression()) + { + if (!$this->EvalBinary($token)) + { + return false; + } + // else continue + } + else + { + // an error must have occurred + return false; + } + break; + default: + --$this->pos; + return true; + } + } + return true; + } + + /* + * Process "op a" where op in (+,-,!) + * @return boolean - true if success, false if any error occurred + / + + private function EvaluateUnaryExpression() + { + if (($this->pos + 1) >= $this->count) { + $this->AddError("Poorly terminated expression - expected a constant or variable", NULL); + return false; + } + $token = $this->tokens[++$this->pos]; + switch ($token[0]) + { + case '+': + case '-': + case '!': + if (!$this->EvaluatePrimaryExpression()) + { + return false; + } + return $this->EvalUnary($token); + break; + default: + --$this->pos; + return $this->EvaluatePrimaryExpression(); + } + } + + /* + * Returns array of all reserved words used when parsing a string via sProcessStringContainingExpressions + * @return <type> + / + + public function GetAllReservedWordsUsed() + { + return array_unique($this->allReservedWordsUsed); + } + + /* + * Returns array of all variables used when parsing a string via sProcessStringContainingExpressions + * @return <type> + / + public function GetAllVarsUsed() + { + return array_unique($this->allVarsUsed); + } + + /* + * Return the result of evaluating the equation - NULL if error + * @return mixed + / + public function GetResult() + { + return $this->result[0]; + } + + /* + * Return an array of errors + * @return array + / + public function GetErrors() + { + return $this->errs; + } + + /* + * Return an array of human-readable errors (message, offending token, offset of offending token within equation) + * @return array + / + public function GetReadableErrors() + { + $errs = array(); + foreach ($this->errs as $err) + { + $msg = $err[0]; + $token = $err[1]; + $toshow = 'ERR'; + if (!is_null($token)) + { + $toshow .= '[' . $token[0] . ' @pos=' . $token[1] . ']'; + } + $toshow .= ': ' . $msg; + $errs[] = $toshow; + } + return $errs; + } + + /* + * Return array of list of reserved words used in the equation + * @return <type> + / + + public function GetReservedWordsUsed() + { + return array_unique($this->reservedWordsUsed); + } + + /* + * Return array of the list of variables used in the equation + * @return array + / + public function GetVarsUsed() + { + return array_unique($this->varsUsed); + } + + /* + * Return true if there were syntax or processing errors + * @return boolean + / + public function HasErrors() + { + return (count($this->errs) > 0); + } + + /* + * Return true if there are syntax errors + * @return boolean + / + private function HasSyntaxErrors() + { + // check for bad tokens + // check for unmatched parentheses + // check for undefined variables + // check for undefined functions (but can't easily check allowable # elements?) + + $nesting = 0; + + for ($i=0;$i<$this->count;++$i) + { + $token = $this->tokens[$i]; + switch ($token[2]) + { + case 'LP': + ++$nesting; + break; + case 'RP': + --$nesting; + if ($nesting < 0) + { + $this->AddError("Extra ')' detected", $token); + } + break; + case 'WORD': + case 'SGQA': + if ($i+1 < $this->count and $this->tokens[$i+1][2] == 'LP') + { + if (!$this->isValidFunction($token[0])) + { + $this->AddError("Undefined function", $token); + } + } + else + { + if (!($this->isValidVariable($token[0]) or $this->isValidReservedWord($token[0]))) + { + $this->AddError("Undefined variable or reserved word", $token); + } + } + break; + case 'OTHER': + $this->AddError("Unsupported syntax", $token); + break; + default: + break; + } + } + if ($nesting != 0) + { + $this->AddError("Parentheses not balanced",NULL); + } + return (count($this->errs) > 0); + } + + /* + * Return true if the function name is registered + * @param <type> $name + * @return boolean + / + + private function isValidFunction($name) + { + return array_key_exists($name,$this->amValidFunctions); + } + + /* + * Return true if the reserved word name is registered + * @param <type> $name + * @return boolean + / + private function isValidReservedWord($name) + { + return array_key_exists($name,$this->amReservedWords); + } + + /* + * Return true if the variable name is registered + * @param <type> $name + * @return boolean + / + private function isValidVariable($name) + { + return array_key_exists($name,$this->amVars); + } + + /* + * Process a full string, containing multiple expressions delimited by {}, return a consolidated string + * @param <type> $src + / + + public function sProcessStringContainingExpressions($src) + { + // tokenize string by the {} pattern, properly dealing with strings in quotations, and escaped curly brace values + $stringParts = $this->asSplitStringOnExpressions($src); + + $resolvedParts = array(); + $this->allVarsUsed = array(); + $this->allReservedWordsUsed = array(); + + foreach ($stringParts as $stringPart) + { + if ($stringPart[2] == 'STRING') { + $resolvedParts[] = $stringPart[0]; + } + else { + if ($this->Evaluate(substr($stringPart[0],1,-1))) + { + $resolvedParts[] = $this->GetResult(); + $this->allVarsUsed = array_merge($this->allVarsUsed,$this->GetVarsUsed()); + $this->allReservedWordsUsed = array_merge($this->allReservedWordsUsed, $this->GetReservedWordsUsed()); + } + else + { + // show original and errors in-line? + $resolvedParts[] = '[' . $stringPart[0] . ':' . implode(';',$this->GetReadableErrors()) . ']'; + } + } + } + $result = implode('',$resolvedParts); + return $result; + } + + /* + * Run a registered function + * @param <type> $funcNameToken + * @param <type> $params + * @return boolean + / + private function RunFunction($funcNameToken,$params) + { + $name = $funcNameToken[0]; + if (!$this->isValidFunction($name)) + { + return false; + } + $func = $this->amValidFunctions[$name]; + $funcName = $func[0]; + $numArgs = count($params); + + if (function_exists($funcName)) { + $numArgsAllowed = array_slice($func, 2); + $argsPassed = is_array($params) ? count($params) : 0; + + // for unlimited # parameters + try + { + if (in_array(-1, $numArgsAllowed)) { + $result = $funcName($params); + + // Call function with the params passed + } elseif (in_array($argsPassed, $numArgsAllowed)) { + + switch ($argsPassed) { + case 0: + $result = $funcName(); + break; + case 1: + $result = $funcName($params[0]); + break; + case 2: + $result = $funcName($params[0], $params[1]); + break; + case 3: + $result = $funcName($params[0], $params[1], $params[2]); + break; + case 4: + $result = $funcName($params[0], $params[1], $params[2], $params[3]); + break; + default: + $this->AddError("Error: Unsupported arg count: $funcName(".implode(", ",$params),$funcNameToken); + return false; + } + + } else { + $this->AddError("Error: Incorrect arg count: " . $funcName ."(".implode(", ",$params).")",$funcNameToken); + return false; + } + } + catch (Exception $e) + { + $this->AddError($e->getMessage(),$funcNameToken); + return false; + } + $token = array($result,$funcNameToken[1],'NUMBER'); + $this->StackPush($token); + return true; + } + } + + /* + * Add user functions to array of allowable functions within the equation. + * $functions is an array of key to value mappings like this: + * 'newfunc' => array('my_func_script', 1,3) + * where 'newfunc' is the name of an allowable function wihtin the expression, 'my_func_script' is the registered PHP function name, + * and 1,3 are the list of allowable numbers of paremeters (so my_func() can take 1 or 3 parameters. + * + * @param array $functions + / + + public function RegisterFunctions(array $functions) { + $this->amValidFunctions= array_merge($this->amValidFunctions, $functions); + } + + /* + * Add list of allowable ReservedWord names within the equation + * $varnames is an array of key to value mappings like this: + * 'myvar' => value + * where value is optional (e.g. can be blank), and can be any scalar type (e.g. string, number, but not array) + * the system will use the values as fast lookup when doing calculations, but if it needs to set values, it will call + * the interface function to set the values by name + * + * @param array $varnames + / + public function RegisterReservedWords(array $varnames) { + $this->amReservedWords = array_merge($this->amReservedWords, $varnames); + } + + /* + * Add list of allowable variable names within the equation + * $varnames is an array of key to value mappings like this: + * 'myvar' => value + * where value is optional (e.g. can be blank), and can be any scalar type (e.g. string, number, but not array) + * the system will use the values as fast lookup when doing calculations, but if it needs to set values, it will call + * the interface function to set the values by name + * + * @param array $varnames + / + public function RegisterVarnames(array $varnames) { + $this->amVars = array_merge($this->amVars, $varnames); + } + + /* + * Set the value of a registered variable + * @param $op - the operator (=,=,/=,+=,-=) + @param <type> $name + * @param <type> $value + / + private function setVariableValue($op,$name,$value) + { + // TODO - set this externally + switch($op) + { + case '=': + $this->amVars[$name] = $value; + break; + case '=': + $this->amVars[$name] = $value; + break; + case '/=': + $this->amVars[$name] /= $value; + break; + case '+=': + $this->amVars[$name] += $value; + break; + case '-=': + $this->amVars[$name] -= $value; + break; + } + return $this->amVars[$name]; + } + + public function asSplitStringOnExpressions($src) + { + // tokenize string by the {} pattern, propertly dealing with strings in quotations, and escaped curly brace values + $tokens0 = preg_split($this->sExpressionRegex,$src,-1,(PREG_SPLIT_NO_EMPTY \| PREG_SPLIT_DELIM_CAPTURE \| PREG_SPLIT_OFFSET_CAPTURE)); + + $tokens = array(); + // Add token_type to $tokens: For each token, test each categorization in order - first match will be the best. + for ($j=0;$j<count($tokens0);++$j) + { + $token = $tokens0[$j]; + if (preg_match($this->sExpressionRegex,$token[0])) + { + $token[2] = 'EXPRESSION'; + } + else + { + $token[2] = 'STRING'; + } + $tokens[] = $token; + } + return $tokens; + } + + /* + * Pop a value token off of the stack + * @return token + / + + private function StackPop() + { + if (count($this->stack) > 0) + { + return array_pop($this->stack); + } + else + { + $this->AddError("Tried to pop value off of empty stack", NULL); + return NULL; + } + } + + /* + * Stack only holds values (number, string), not operators + * @param array $token + / + + private function StackPush(array $token) + { + if ($this->onlyparse) + { + // If only parsing, still want to validate syntax, so use "1" for all variables + switch($token[2]) + { + case 'STRING': + $this->stack[] = array(1,$token[1],'STRING'); + break; + case 'NUMBER': + default: + $this->stack[] = array(1,$token[1],'NUMBER'); + break; + } + } + else + { + $this->stack[] = $token; + } + } + + /* + * Split the source string into tokens, removing whitespace, and categorizing them by type. + * + * @param $src + * @return array + / + + private function amTokenize($src) + { + // $tokens0 = array of tokens from equation, showing value and offset position. Will include SPACE, which should be removed + $tokens0 = preg_split($this->sTokenizerRegex,$src,-1,(PREG_SPLIT_NO_EMPTY \| PREG_SPLIT_DELIM_CAPTURE \| PREG_SPLIT_OFFSET_CAPTURE)); + + // $tokens = array of tokens from equation, showing value, offsete position, and type. Will not contain SPACE, but will contain OTHER + $tokens = array(); + // Add token_type to $tokens: For each token, test each categorization in order - first match will be the best. + for ($j=0;$j<count($tokens0);++$j) + { + for ($i=0;$i<count($this->asCategorizeTokensRegex);++$i) + { + $token = $tokens0[$j][0]; + if (preg_match($this->asCategorizeTokensRegex[$i],$token)) + { + if ($this->asTokenType[$i] !== 'SPACE') { + $tokens0[$j][2] = $this->asTokenType[$i]; + if ($this->asTokenType[$i] == 'STRING') + { + // remove outside quotes + $unquotedToken = stripslashes(substr($token,1,-1)); + $tokens0[$j][0] = $unquotedToken; + } + $tokens[] = $tokens0[$j]; // get first matching non-SPACE token type and push onto $tokens array + } + break; // only get first matching token type + } + } + } + return $tokens; + } + + /* + * Unit test the asSplitStringOnExpressions() function to ensure that accurately parses out all expressions + * surrounded by curly braces, allowing for strings and escaped curly braces. + / + + static function UnitTestStringSplitter() + { + $tests = <<<EOD +"this is a string that contains {something in curly braces)" +This example has escaped curly braces like \{this is not an equation\} +Should the parser check for unmatched { opening curly braces? +What about for unmatched } closing curly braces? +{ANS:name}, you said that you are {ANS:age} years old, and that you have {ANS:numKids} {EVAL:if((numKids==1),'child','children')} and {ANS:numPets} {EVAL:if((numPets==1),'pet','pets')} running around the house. So, you have {EVAL:numKids + numPets} wild {EVAL:if((numKids + numPets ==1),'beast','beasts')} to chase around every day. +Since you have more {EVAL:if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'children','pets')} than you do {EVAL:if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')}, do you feel that the {EVAL:if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')} are at a disadvantage? +EOD; + + $em = new ExpressionManager(); + + foreach(explode("\n",$tests) as $test) + { + $tokens = $em->asSplitStringOnExpressions($test); + print '<b>' . $test . '</b><hr/>'; + print '<code>'; + print implode("<br/>\n",explode("\n",print_r($tokens,TRUE))); + print '</code><hr/>'; + } + } + + /* + * Unit test the Tokenizer - Tokenize and generate a HTML-compatible print-out of a comprehensive set of test cases + / + + static function UnitTestTokenizer() + { + // Comprehensive test cases for tokenizing + $tests = <<<EOD + String: "Can strings contain embedded \"quoted passages\" (and parenthesis + other characters?)?" + String: "can single quoted strings" . 'contain nested \'quoted sections\'?'; + Parens: upcase('hello'); + Numbers: 42 72.35 -15 +37 42A .5 0.7 + And_Or: (this and that or the other); Sandles, sorting; (a && b \|\| c) + Words: hi there, my name is C3PO! + UnaryOps: ++a, --b !b + BinaryOps: (a + b c / d) + Comparators: > >= < <= == != gt ge lt le eq ne (target large gents built agile less equal) + Assign: = += -= = /= + SGQA: 1X6X12 1X6X12ber1 1X6X12ber1_lab1 3583X84X249 + Errors: Apt # 10C; (2 > 0) ? 'hi' : 'there'; array[30]; >>> <<< / this is not a comment / // neither is this +EOD; + + $em = new ExpressionManager(); + + foreach(explode("\n",$tests) as $test) + { + $tokens = $em->amTokenize($test); + print '<b>' . $test . '</b><hr/>'; + print '<code>'; + print implode("<br/>\n",explode("\n",print_r($tokens,TRUE))); + print '</code><hr/>'; + } + } + + /* + * Unit test the Evaluator, allowing for passing in of extra functions, variables, and tests + * @param array $extraFunctions + * @param array $extraVars + * @param <type> $extraTests + / + + static function UnitTestEvaluator(array $extraFunctions=array(), array $extraVars=array(), $extraTests='1~1') + { + // Some test cases for Evaluator + $vars = array( + 'one' =>1, + 'two' =>2, + 'three' =>3, + 'four' =>4, + 'five' =>5, + 'six' =>6, + 'seven' =>7, + 'eight' =>8, + 'nine' =>9, + 'ten' =>10, + 'eleven' => 11, + 'twelve' => 12, + 'half' =>.5, + 'hi' =>'there', + 'hello' =>"Tom", + 'a' =>0, + 'b' =>0, + 'c' =>0, + 'd' =>0, + '12X34X56' =>5, + '12X3X5lab1_ber' =>10, + ); + + $reservedWord = array( + 'ADMINEMAIL' =>'{ADMINEMAIL}', + 'ADMINNAME' =>'{ADMINNAME}', + 'AID' =>'{AID}', + 'ANSWERSCLEARED' =>'{ANSWERSCLEARED}', + 'ANSWER' =>'{ANSWER}', + 'ASSESSMENTS' =>'{ASSESSMENTS}', + 'ASSESSMENT_CURRENT_TOTAL' =>'{ASSESSMENT_CURRENT_TOTAL}', + 'ASSESSMENT_HEADING' =>'{ASSESSMENT_HEADING}', + 'CHECKJAVASCRIPT' =>'{CHECKJAVASCRIPT}', + 'CLEARALL' =>'{CLEARALL}', + 'CLOSEWINDOW' =>'{CLOSEWINDOW}', + 'COMPLETED' =>'{COMPLETED}', + 'DATESTAMP' =>'{DATESTAMP}', + 'EMAILCOUNT' =>'{EMAILCOUNT}', + 'EMAIL' =>'{EMAIL}', + 'EXPIRY' =>'{EXPIRY}', + 'FIRSTNAME' =>'{FIRSTNAME}', + 'GID' =>'{GID}', + 'GROUPDESCRIPTION' =>'{GROUPDESCRIPTION}', + 'GROUPNAME' =>'{GROUPNAME}', + 'INSERTANS:123X45X67' =>'{INSERTANS:123X45X67}', + 'INSERTANS:123X45X67ber' =>'{INSERTANS:123X45X67ber}', + 'INSERTANS:123X45X67ber_01a' =>'{INSERTANS:123X45X67ber_01a}', + 'LANGUAGECHANGER' =>'{LANGUAGECHANGER}', + 'LANGUAGE' =>'{LANGUAGE}', + 'LANG' =>'{LANG}', + 'LASTNAME' =>'{LASTNAME}', + 'LOADERROR' =>'{LOADERROR}', + 'LOADFORM' =>'{LOADFORM}', + 'LOADHEADING' =>'{LOADHEADING}', + 'LOADMESSAGE' =>'{LOADMESSAGE}', + 'NAME' =>'{NAME}', + 'NAVIGATOR' =>'{NAVIGATOR}', + 'NOSURVEYID' =>'{NOSURVEYID}', + 'NOTEMPTY' =>'{NOTEMPTY}', + 'NULL' =>'{NULL}', + 'NUMBEROFQUESTIONS' =>'{NUMBEROFQUESTIONS}', + 'OPTOUTURL' =>'{OPTOUTURL}', + 'PASSTHRULABEL' =>'{PASSTHRULABEL}', + 'PASSTHRUVALUE' =>'{PASSTHRUVALUE}', + 'PERCENTCOMPLETE' =>'{PERCENTCOMPLETE}', + 'PERC' =>'{PERC}', + 'PRIVACYMESSAGE' =>'{PRIVACYMESSAGE}', + 'PRIVACY' =>'{PRIVACY}', + 'QID' =>'{QID}', + 'QUESTIONHELPPLAINTEXT' =>'{QUESTIONHELPPLAINTEXT}', + 'QUESTIONHELP' =>'{QUESTIONHELP}', + 'QUESTION_CLASS' =>'{QUESTION_CLASS}', + 'QUESTION_CODE' =>'{QUESTION_CODE}', + 'QUESTION_ESSENTIALS' =>'{QUESTION_ESSENTIALS}', + 'QUESTION_FILE_VALID_MESSAGE' =>'{QUESTION_FILE_VALID_MESSAGE}', + 'QUESTION_HELP' =>'{QUESTION_HELP}', + 'QUESTION_INPUT_ERROR_CLASS' =>'{QUESTION_INPUT_ERROR_CLASS}', + 'QUESTION_MANDATORY' =>'{QUESTION_MANDATORY}', + 'QUESTION_MAN_CLASS' =>'{QUESTION_MAN_CLASS}', + 'QUESTION_MAN_MESSAGE' =>'{QUESTION_MAN_MESSAGE}', + 'QUESTION_NUMBER' =>'{QUESTION_NUMBER}', + 'QUESTION_TEXT' =>'{QUESTION_TEXT}', + 'QUESTION_VALID_MESSAGE' =>'{QUESTION_VALID_MESSAGE}', + 'QUESTION' =>'{QUESTION}', + 'REGISTERERROR' =>'{REGISTERERROR}', + 'REGISTERFORM' =>'{REGISTERFORM}', + 'REGISTERMESSAGE1' =>'{REGISTERMESSAGE1}', + 'REGISTERMESSAGE2' =>'{REGISTERMESSAGE2}', + 'RESTART' =>'{RESTART}', + 'RETURNTOSURVEY' =>'{RETURNTOSURVEY}', + 'SAVEALERT' =>'{SAVEALERT}', + 'SAVEDID' =>'{SAVEDID}', + 'SAVEERROR' =>'{SAVEERROR}', + 'SAVEFORM' =>'{SAVEFORM}', + 'SAVEHEADING' =>'{SAVEHEADING}', + 'SAVEMESSAGE' =>'{SAVEMESSAGE}', + 'SAVE' =>'{SAVE}', + 'SGQ' =>'{SGQ}', + 'SID' =>'{SID}', + 'SITENAME' =>'{SITENAME}', + 'SUBMITBUTTON' =>'{SUBMITBUTTON}', + 'SUBMITCOMPLETE' =>'{SUBMITCOMPLETE}', + 'SUBMITREVIEW' =>'{SUBMITREVIEW}', + 'SURVEYCONTACT' =>'{SURVEYCONTACT}', + 'SURVEYDESCRIPTION' =>'{SURVEYDESCRIPTION}', + 'SURVEYFORMAT' =>'{SURVEYFORMAT}', + 'SURVEYLANGAGE' =>'{SURVEYLANGAGE}', + 'SURVEYLISTHEADING' =>'{SURVEYLISTHEADING}', + 'SURVEYLIST' =>'{SURVEYLIST}', + 'SURVEYNAME' =>'{SURVEYNAME}', + 'SURVEYURL' =>'{SURVEYURL}', + 'TEMPLATECSS' =>'{TEMPLATECSS}', + 'TEMPLATEURL' =>'{TEMPLATEURL}', + 'TEXT' =>'{TEXT}', + 'THEREAREXQUESTIONS' =>'{THEREAREXQUESTIONS}', + 'TIME' =>'{TIME}', + 'TOKEN:EMAIL' =>'{TOKEN:EMAIL}', + 'TOKEN:FIRSTNAME' =>'{TOKEN:FIRSTNAME}', + 'TOKEN:LASTNAME' =>'{TOKEN:LASTNAME}', + 'TOKEN:XXX' =>'{TOKEN:XXX}', + 'TOKENCOUNT' =>'{TOKENCOUNT}', + 'TOKEN_COUNTER' =>'{TOKEN_COUNTER}', + 'TOKEN' =>'{TOKEN}', + 'URL' =>'{URL}', + 'WELCOME' =>'{WELCOME}', + ); + + // Syntax for $tests is~ + // expectedResult~expression + // if the expected result is an error, use NULL for the expected result + $tests = <<<EOD +50~12X34X56 12X3X5lab1_ber +3~a=three +3~c=a +12~c=four +15~c+=a +5~c/=a +-1~c-=six +2~max(one,two) +5~max(one,two,three,four,five) +1024~max(one,(twothree),pow(four,five),six) +1~min(one,two,three,four,five) +27~pow(3,3) +5~hypot(three,four) +0~0 +24~one * two * three * four +-4~five - four - three - two +0~two * three - two - two - two +4~two * three - two +3.1415926535898~pi() +1~pi() == pi() * 2 - pi() +1~sin(pi()/2) +1~sin(0.5 * pi()) +1~sin(pi()/2) == sin(.5 * pi()) +105~5 + 1, 7 * 15 +7~7 +15~10 + 5 +24~12 * 2 +10~13 - 3 +3.5~14 / 4 +5~3 + 1 * 2 +1~one +there~hi +6.25~one * two - three / four + five +1~one + hi +1~two > one +1~two gt one +1~three >= two +1~three ge two +0~four < three +0~four lt three +0~four <= three +0~four le three +0~four == three +0~four eq three +1~four != three +0~four ne four +0~one * hi +5~abs(-five) +0~acos(pi()/2) +0~asin(pi()/2) +10~ceil(9.1) +9~floor(9.9) +32767~getrandmax() +0~rand() +15~sum(one,two,three,four,five) +5~intval(5.7) +1~is_float('5.5') +0~is_float('5') +1~is_numeric(five) +0~is_numeric(hi) +1~is_string(hi) +2.4~(one * two) + (three * four) / (five * six) +1~(one * (two + (three - four) + five) / six) +0~one && 0 +0~two and 0 +1~five && 6 +1~seven && eight +1~one or 0 +1~one \|\| 0 +1~(one and 0) \|\| (two and three) +NULL~hi(there); +NULL~(one * two + (three - four) +NULL~(one * two + (three - four))) +NULL~++a +NULL~--b +11~eleven +144~twelve * twelve +4~if(5 > 7,2,4) +there~if((one > two),'hi','there') +64~if((one < two),pow(2,6),pow(6,2)) +1,2,3,4,5~list(one,two,three,min(four,five,six),max(three,four,five)) +11,12~list(eleven,twelve) +{INSERTANS:123X45X67}~INSERTANS:123X45X67 +{QID}~QID +{ASSESSMENT_HEADING}~ASSESSMENT_HEADING +{TOKEN:FIRSTNAME}~TOKEN:FIRSTNAME +{THEREAREXQUESTIONS}~THEREAREXQUESTIONS +EOD; + + $em = new ExpressionManager(); + $em->RegisterVarnames($vars); + $em->RegisterReservedWords($reservedWord); + + if (is_array($extraVars) and count($extraVars) > 0) + { + $em->RegisterVarnames($extraVars); + } + if (is_array($extraFunctions) and count($extraFunctions) > 0) + { + $em->RegisterFunctions($extraFunctions); + } + if (is_string($extraTests)) + { + $tests .= "\n" . $extraTests; + } + + print '<table border="1"><tr><th>Expression</th><th>Result</th><th>Expected</th><th>VarsUsed</th><th>ReservedWordsUsed</th><th>Errors</th></tr>'; + foreach(explode("\n",$tests)as $test) + { + $values = explode("~",$test); + $expectedResult = array_shift($values); + $expr = implode("~",$values); + $resultStatus = 'ok'; + print '<tr><td>' . $expr . "</td>\n"; + $status = $em->Evaluate($expr); + $result = $em->GetResult(); + $valToShow = $result; + if (is_null($result)) { + $valToShow = "NULL"; + } + print '<td>' . $valToShow . "</td>\n"; + if ($valToShow != $expectedResult) + { + $resultStatus = 'error'; + } + print "<td class='" . $resultStatus . "'>" . $expectedResult . "</td>\n"; + $varsUsed = $em->GetVarsUsed(); + if (is_array($varsUsed) and count($varsUsed) > 0) { + print '<td>' . implode(', ', $varsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $reservedWordsUsed = $em->GetReservedWordsUsed(); + if (is_array($reservedWordsUsed) and count($reservedWordsUsed) > 0) { + print '<td>' . implode(', ', $reservedWordsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $errs = $em->GetReadableErrors(); + $errStatus = 'ok'; + if (is_array($errs) and count($errs) > 0) { + if ($expectedResult != "NULL") + { + $errStatus = 'error'; // should have been error free + } + print "<td class='" . $errStatus . "'>" . implode("<br/>\n", $errs) . "</td>\n"; + } + else { + if ($expectedResult == "NULL") + { + $errStatus = 'error'; // should have had errors + } + print "<td class='" . $errStatus . "'> </td>\n"; + } + print '</tr>'; + } + print '</table>'; + } + + static function UnitTestProcessStringContainingExpressions() + { + $vars = array( + 'name' => 'Sergei', + 'age' => 45, + 'numKids' => 2, + 'numPets' => 1, + ); + $reservedWords = array( + 'INSERTANS:61764X1X1' => 'Peter', + 'INSERTANS:61764X1X2' => 27, + 'INSERTANS:61764X1X3' => 1, + 'INSERTANS:61764X1X4' => 8 + ); + + $tests = <<<EOD +{name}, you said that you are {age} years old, and that you have {numKids} {if((numKids==1),'child','children')} and {numPets} {if((numPets==1),'pet','pets')} running around the house. So, you have {numKids + numPets} wild {if((numKids + numPets ==1),'beast','beasts')} to chase around every day. +Since you have more {if((numKids > numPets),'children','pets')} than you do {if((numKids > numPets),'pets','children')}, do you feel that the {if((numKids > numPets),'pets','children')} are at a disadvantage? +{INSERTANS:61764X1X1}, you said that you are {INSERTANS:61764X1X2} years old, and that you have {INSERTANS:61764X1X3} {if((INSERTANS:61764X1X3==1),'child','children')} and {INSERTANS:61764X1X4} {if((INSERTANS:61764X1X4==1),'pet','pets')} running around the house. So, you have {INSERTANS:61764X1X3 + INSERTANS:61764X1X4} wild {if((INSERTANS:61764X1X3 + INSERTANS:61764X1X4 ==1),'beast','beasts')} to chase around every day. +Since you have more {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'children','pets')} than you do {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')}, do you feel that the {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')} are at a disadvantage? +EOD; + + $em = new ExpressionManager(); + $em->RegisterVarnames($vars); + $em->RegisterReservedWords($reservedWords); + + print '<table border="1"><tr><th>Test</th><th>Result</th><th>VarsUsed</th><th>ReservedWordsUsed</th></tr>'; + foreach(explode("\n",$tests) as $test) + { + print "<tr><td>" . $test . "</td>\n"; + print "<td>" . $em->sProcessStringContainingExpressions($test) . "</td>\n"; + $allVarsUsed = $em->getAllVarsUsed(); + if (is_array($allVarsUsed) and count($allVarsUsed) > 0) { + print "<td>" . implode(', ', $allVarsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $allReservedWordsUsed = $em->getAllReservedWordsUsed(); + if (is_array($allReservedWordsUsed) and count($allReservedWordsUsed) > 0) { + print "<td>" . implode(', ', $allReservedWordsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + print "</tr>\n"; + } + print '</table>'; + } +} + +/* + * Extra Functions can go here. TODO: Find good way to inlcude these extra functions externally. + * Tried via ExpressionManagerFunctions, but they weren't properly included in dFunctionEval.php + / + +function exprmgr_if($test,$ok,$error) +{ + if ($test) + { + return $ok; + } + else + { + return $error; + } +} + +function exprmgr_list($args) +{ + return implode(",",$args); +} + + +?> Index: classes/eval/LimeExpressionManager.php --- classes/eval/LimeExpressionManager.php Locally New +++ classes/eval/LimeExpressionManager.php Locally New @@ -0,0 +1,230 @@ +<?php +/* + * Description of LimeExpressionManager + * This is a wrapper class around ExpressionManager that implements a Singleton and eases + * passing of LimeSurvey variable values into ExpressionManager + * + * @author Thomas M. White + / +include_once('ExpressionManager.php'); + +class LimeExpressionManager { + private static $instance; + private $fieldmap; + private $varMap; + private $sgqaMap; + private $tokenMap; + + private $em; // Expression Manager + + // A private constructor; prevents direct creation of object + private function __construct() + { + $this->em = new ExpressionManager(); + } + + // The singleton method + public static function singleton() + { + if (!isset(self::$instance)) { + $c = __CLASS__; + self::$instance = new $c; + } + return self::$instance; + } + + // Prevent users to clone the instance + public function __clone() + { + trigger_error('Clone is not allowed.', E_USER_ERROR); + } + + /* + * Create the arrays needed by ExpressionManager to process LimeSurvey strings. + * The long part of this function should only be called once per page display (e.g. only if $fieldMap changes) + * + * @param <type> $sid + * @param <type> $forceRefresh + * @return boolean - true if $fieldmap had been re-created, so ExpressionManager variables need to be re-set + / + + public function setVariableAndTokenMappingsForExpressionManager($sid,$forceRefresh=false) + { + global $globalfieldmap, $clang; + + //checks to see if fieldmap has already been built for this page. + if (isset($globalfieldmap[$sid]['full'][$clang->langcode])) { + if (isset($this->fieldmap) && !$forceRefresh) { + return false; // means the mappings have already been set and don't need to be re-created + } + } + + $fieldmap=createFieldMap($sid,$style='full'); + $this->fieldmap = $fieldmap; + if (!isset($fieldmap)) { + return false; + } + + $knownVars = array(); // mapping of VarName to Value + $knownSGQAs = array(); // mapping of SGQA to Value + foreach($fieldmap as $fielddata) + { + $code = $fielddata['fieldname']; + if (preg_match('#^\d+X\d+X#',$code)) + { + switch($fielddata['type']) + { + case '!': //List - dropdown + case '5': //5 POINT CHOICE radio-buttons + case 'D': //DATE + case 'G': //GENDER drop-down list + case 'I': //Language Question + case 'L': //LIST drop-down/radio-button list + case 'N': //NUMERICAL QUESTION TYPE + case 'O': //LIST WITH COMMENT drop-down/radio-button list + textarea + case 'S': //SHORT FREE TEXT + case 'T': //LONG FREE TEXT + case 'U': //HUGE FREE TEXT + case 'X': //BOILERPLATE QUESTION + case 'Y': //YES/NO radio-buttons + case '\|': //File Upload + $varName = $fielddata['title']; + $question = $fielddata['question']; + break; + case '1': //Array (Flexible Labels) dual scale + $varName = $fielddata['title'] . '.' . $fielddata['aid'] . '.' . $fielddata['scale_id']; + $question = $fielddata['question'] . ': ' . $fielddata['subquestion'] . ': ' . $fielddata['scale']; + break; + case 'A': //ARRAY (5 POINT CHOICE) radio-buttons + case 'B': //ARRAY (10 POINT CHOICE) radio-buttons + case 'C': //ARRAY (YES/UNCERTAIN/NO) radio-buttons + case 'E': //ARRAY (Increase/Same/Decrease) radio-buttons + case 'F': //ARRAY (Flexible) - Row Format + case 'H': //ARRAY (Flexible) - Column Format + case 'K': //MULTIPLE NUMERICAL QUESTION + case 'M': //Multiple choice checkbox + case 'P': //Multiple choice with comments checkbox + text + case 'Q': //MULTIPLE SHORT TEXT + case 'R': //RANKING STYLE + $varName = $fielddata['title'] . '.' . $fielddata['aid']; + $question = $fielddata['question'] . ': ' . $fielddata['subquestion']; + break; + case ':': //ARRAY (Multi Flexi) 1 to 10 + case ';': //ARRAY (Multi Flexi) Text + $varName = $fielddata['title'] . '.' . $fielddata['aid']; + $question = $fielddata['question'] . ': ' . $fielddata['subquestion1'] . ': ' . $fielddata['subquestion2']; + break; + } + } + if (isset($_SESSION[$code])) + { + $codeValue = $_SESSION[$code]; + $displayValue= retrieve_Answer($code, $_SESSION['dateformats']['phpdate']); + $knownVars[$varName] = $codeValue; + $knownVars[$varName . '.shown'] = $displayValue; + $knownVars[$varName . '.question']= $question; + $knownSGQAs['INSERTANS:' . $code] = $displayValue; + } + } + $this->varMap = $knownVars; + $this->sgqaMap = $knownSGQAs; + + // Now set tokens + $tokens = array(); // mapping of TOKENS to values - how often does this need to be set? + if (isset($_SESSION['token']) && $_SESSION['token'] != '') + { + //Gather survey data for tokenised surveys, for use in presenting questions + $_SESSION['thistoken']=getTokenData($surveyid, $_SESSION['token']); + } + if (isset($_SESSION['thistoken'])) + { + // TODO - need to explicitly set TOKEN:FIRSTNAME, and related to blank if not using tokens? + foreach (array_keys($_SESSION['thistoken']) as $tokenkey) + { + $tokens["TOKEN:" . strtoupper($tokenkey)] = $_SESSION['thistoken'][$tokenkey]; + } + } + $this->tokenMap = $tokens; + + return true; + } + + /* + * Translate all Expressions, Macros, registered variables, etc. in $string + * @param <type> $string + * @param <type> $sid + * @param boolean $forceRefresh - if true, reset $fieldMap and the derived arrays of registered variables and values + * @return string - the original $string with all replacements done. + / + + static function ProcessString($string,$sid,$forceRefresh=false) + { + $lem = LimeExpressionManager::singleton(); + $em = $lem->em; + if ($lem->setVariableAndTokenMappingsForExpressionManager($sid,$forceRefresh)) + { + // means that some values changed, so need to update what was registered to ExpressionManager + $em->RegisterVarnamesUsingReplace($lem->varMap); + $em->RegisterReservedWordsUsingReplace($lem->sgqaMap); + $em->RegisterReservedWordsUsingMerge($lem->tokenMap); + } + return $em->sProcessStringContainingExpressions(htmlspecialchars_decode($string)); + } + + + /* + * Unit test + / + static function UnitTestProcessStringContainingExpressions() + { + $vars = array( + 'name' => 'Sergei', + 'age' => 45, + 'numKids' => 2, + 'numPets' => 1, + ); + $reservedWords = array( + 'INSERTANS:61764X1X1' => 'Peter', + 'INSERTANS:61764X1X2' => 27, + 'INSERTANS:61764X1X3' => 1, + 'INSERTANS:61764X1X4' => 8 + ); + + $tests = <<<EOD +{name}, you said that you are {age} years old, and that you have {numKids} {if((numKids==1),'child','children')} and {numPets} {if((numPets==1),'pet','pets')} running around the house. So, you have {numKids + numPets} wild {if((numKids + numPets ==1),'beast','beasts')} to chase around every day. +Since you have more {if((numKids > numPets),'children','pets')} than you do {if((numKids > numPets),'pets','children')}, do you feel that the {if((numKids > numPets),'pets','children')} are at a disadvantage? +{INSERTANS:61764X1X1}, you said that you are {INSERTANS:61764X1X2} years old, and that you have {INSERTANS:61764X1X3} {if((INSERTANS:61764X1X3==1),'child','children')} and {INSERTANS:61764X1X4} {if((INSERTANS:61764X1X4==1),'pet','pets')} running around the house. So, you have {INSERTANS:61764X1X3 + INSERTANS:61764X1X4} wild {if((INSERTANS:61764X1X3 + INSERTANS:61764X1X4 ==1),'beast','beasts')} to chase around every day. +Since you have more {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'children','pets')} than you do {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')}, do you feel that the {if((INSERTANS:61764X1X3 > INSERTANS:61764X1X4),'pets','children')} are at a disadvantage? +EOD; + + $lem = LimeExpressionManager::singleton(); + $em = $lem->em; + + $em->RegisterVarnamesUsingMerge($vars); + $em->RegisterReservedWordsUsingMerge($reservedWords); + + print '<table border="1"><tr><th>Test</th><th>Result</th><th>VarsUsed</th><th>ReservedWordsUsed</th></tr>'; + foreach(explode("\n",$tests) as $test) + { + print "<tr><td>" . $test . "</td>\n"; + print "<td>" . $em->sProcessStringContainingExpressions($test) . "</td>\n"; + $allVarsUsed = $em->getAllVarsUsed(); + if (is_array($allVarsUsed) and count($allVarsUsed) > 0) { + print "<td>" . implode(', ', $allVarsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + $allReservedWordsUsed = $em->getAllReservedWordsUsed(); + if (is_array($allReservedWordsUsed) and count($allReservedWordsUsed) > 0) { + print "<td>" . implode(', ', $allReservedWordsUsed) . "</td>\n"; + } + else { + print "<td> </td>\n"; + } + print "</tr>\n"; + } + print '</table>'; + } +} +?> Index: classes/eval/Test_ExpressionManager_Evaluate.php --- classes/eval/Test_ExpressionManager_Evaluate.php Locally New +++ classes/eval/Test_ExpressionManager_Evaluate.php Locally New @@ -0,0 +1,29 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <style type="text/css"> + <!-- +.error { + background-color: #ff0000; +} +.ok { + background-color: #00ff00 +} + --> + </style> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; +// include 'ExpressionManagerFunctions.php'; +// ExpressionManager::UnitTestEvaluator($exprmgr_functions,$exprmgr_extraVars,$exprmgr_extraTests); + ExpressionManager::UnitTestEvaluator(); + ?> + </body> +</html> Index: classes/eval/Test_ExpressionManager_ProcessStringContainingExpressions.php --- classes/eval/Test_ExpressionManager_ProcessStringContainingExpressions.php Locally New +++ classes/eval/Test_ExpressionManager_ProcessStringContainingExpressions.php Locally New @@ -0,0 +1,17 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; + ExpressionManager::UnitTestProcessStringContainingExpressions(); + ?> + </body> +</html> Index: classes/eval/Test_ExpressionManager_Tokenizer.php --- classes/eval/Test_ExpressionManager_Tokenizer.php Locally New +++ classes/eval/Test_ExpressionManager_Tokenizer.php Locally New @@ -0,0 +1,18 @@ +<!-- +To change this template, choose Tools \| Templates +and open the template in the editor. +--> +<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<html> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <title></title> + </head> + <body> + <?php + include 'ExpressionManager.php'; + ExpressionManager::UnitTestTokenizer(); + ExpressionManager::UnitTestStringSplitter(); + ?> + </body> +</html> Index: common_functions.php --- common_functions.php Base (BASE) +++ common_functions.php Locally Modified (Based On LOCAL) @@ -15,6 +15,7 @@ / +include_once('/classes/eval/LimeExpressionManager.php'); /** * This function gives back an array that defines which survey permissions and what part of the CRUD+Import+Export subpermissions is available. @@ -3147,7 +3148,7 @@ if (strpos($line, "{QUESTIONHELPPLAINTEXT}") !== false) $line=str_replace("{QUESTIONHELPPLAINTEXT}", strip_tags(addslashes($help)), $line); } - $line=insertansReplace($line); +// $line=insertansReplace($line); if (strpos($line, "{SUBMITCOMPLETE}") !== false) $line=str_replace("{SUBMITCOMPLETE}", "<strong>".$clang->gT("Thank you!")."<br /><br />".$clang->gT("You have completed answering the questions in this survey.")."</strong><br /><br />".$clang->gT("Click on 'Submit' now to complete the process and save your answers."), $line); if (strpos($line, "{SUBMITREVIEW}") !== false) { @@ -3160,8 +3161,10 @@ $line=str_replace("{SUBMITREVIEW}", $strreview, $line); } - $line=tokenReplace($line); +// $line=tokenReplace($line); + $line = LimeExpressionManager::ProcessString($line, $surveyid); + if (strpos($line, "{ANSWERSCLEARED}") !== false) $line=str_replace("{ANSWERSCLEARED}", $clang->gT("Answers Cleared"), $line); if (strpos($line, "{RESTART}") !== false) { issue_05103-20110616a.patch (90,279 bytes)

TMSWhite 2011-06-16 14:12 reporter ~15455	Some have asked how to test the ExpressionManager. Here are the details: (1) Apply patch_issue_05103_20110616a.patch (or merge in the ExpressionManager-source.zip) - everything is in the /classes/eval/*, except fora one line change in /classes/dTexts/dTexts.php, plus a 3 line change in common_functions.php (2) Load the ExpressionManager-plus-all-question-types.lss survey. ExpressionManager-screenshots.zip shows screen shots of what it should look like. (3) For lower-level testing, try (a) http://localhost/limesurvey/classes/eval/Test_ExpressionManager_Tokenizer.php (d) http://localhost/limesurvey/classes/eval/Test_ExpressionManager_Evaluate.php (c) http://localhost/limesurvey/classes/eval/Test_ExpressionManager_ProcessStringContainingExpressions.php The screens hots and survey are in ExpressionManager-screenshots.zip The changed source is in ExpressionManager-source.zip

TMSWhite 2011-06-16 14:12 reporter	ExpressionMaanger-Screenshots.zip (450,029 bytes)

TMSWhite 2011-06-16 14:12 reporter	ExpressionManager-source.zip (117,240 bytes)

TMSWhite 2011-06-17 08:28 reporter ~15472	Improved error reporting. Whenever there are expressions with syntax errors, ExpressionManager now highlights the whole expression, red-boxes the text that caused errors (like unknown variables or extra parentheses), and provides mouse-over ToolTips describing the errors.

TMSWhite 2011-06-17 12:00 reporter	issue_05268-20110617.patch (153,393 bytes)

TMSWhite 2011-06-17 12:01 reporter ~15474	To the best of my knowledge, ExpressionManager is now fully operational, and integrated into LimeSurvey via this patch: issue_05268-20110617.patch (see also issue 05268) Please test.

TMSWhite 2011-06-17 12:29 reporter ~15476	For screen shots of ExpressionManager's new way of reporting syntax errors, see here: http://www.limesurvey.org/en/forum/development/62306-gui-for-complex-expressions

TMSWhite 2011-06-18 00:32 reporter ~15483	Have now tested ExpressionManager integration into LimeSurvey for the following features: (1) Development + deployed surveys (2) Token management (3) Template management (4) Data entry screen (5) Printable summary of responses given - passed this through templatereplace() so that can see the text as the respondent saw it, not as the survey-designer built it. In the process, identified 1 small bug in ExpressionManager, and several bugs in other files (mostly lack of detection of undefined variables). The only sections I have yet to test are ones that aren't built into any of my test surveys: (1) Assessments (2) Conditions Please try the latest patch (issue_50268-20110617-v2.patch)

TMSWhite 2011-06-18 00:35 reporter	issue_05268-20110617-v2.patch (155,683 bytes)

TMSWhite 2011-06-19 18:11 reporter ~15509	I need to try some example surveys that have custom JavaScript to ensure that ExpressionManager processes them correctly. Current, ExpressionManager first looks for strings (surrounded by single or double quotes), and then looks for tokens. So, tokens contained within strings are not parsed. Also, it looks for anything within curly braces, so it would think that JavaScript code is an Expression. From the LimeSurvey docs (http://docs.limesurvey.org/Adding+a+question&structure=English+Instructions+for+LimeSurvey), there is this example: Hello {TOKEN:FIRSTNAME}. We sent an email to you using this address {TOKEN:EMAIL}. Is this correct? What do you as a {TOKEN:ATTRIBUTE_1} think about your <script type="text/javascript" language="Javascript">; var job='{TOKEN:ATTRIBUTE_1}'; if (job=='worker') { document.write ('bosses') } else { document.write('workers') } </script> ? ExpressionManager, as is, would think that '{TOKEN:ATTRIBUTE_1}' is a string, and would detect two syntax errors here: if (job=='worker') { document.write ('bosses') } else { document.write('workers') } Some solutions may include: (1) Conditionally parse expressions within strings (2) Ignore curly braces with this pattern '{\s+\n.?\s}' - this would put the burden on the survey authors to not write JavaScript code like if (job=='worker') { document.write('bosses') } else { document.write('workers') } This also might get a little tricky in detecting nested curly braces so that it matches the correct pair - so this should be an optional parsing style just for people wanting to use JavaScript overrides. (3) Ignore anything between XML quotes (<!-- -->) - but that wouldn't allow substitution within JavaScript strings like '{TOKEN:ATTRIBUTE_1}' However, that might support recursive substitution without getting confused by JavaScript added as part of a substitution step. Thoughts?

TMSWhite 2011-07-02 07:19 reporter ~15614	Fixed ExpressionManager so that it properly substitutes {TOKEN:ATTRIBUTE_1} above without throwing errors related to the JavaScript code

TMSWhite 2011-07-04 09:36 reporter ~15626	ExpressionManager is now stable in the limesurvey_dev_tms branch Use the following to test it: (1) http://localhost/limesurvey_dev_tms/classes/eval/ExpressionManagerTestSuite.php (2) Load and test this survey: ExpressionManager-plus-all-question-types.lss

TMSWhite 2011-07-11 23:22 reporter ~15731	Syntax changed slightly to so that JavaScript and CSS can be safely handled. ExpressionManager will now only process expressions that are surrounded by curly braces with no leading or trailing white space. So: {5 7} => 35 { 5 7} => { 5 7} {5 7 } => {5 * 7 } There are a few places within qanda.php where JavaScript functions did not have any leading or trailing spaces (e.g. {document.getElementById()....}), but now adding a single space will keep them from being highlighted by ExpressionManager as having syntax errors. This may require that people with custom JavaScript ensure that they have added such leading/trailing spaces, or just use line-feed to properly indent their code. I expect this will be the minority of users. There may be a way to detect such custom JavaScript at survey load or activation time. To test this, try latest limesurvey_dev_tms revision (10481) using the RegressionTester-en-de.lss instrument

TMSWhite 2011-07-11 23:25 reporter	RegressionTester-en-de.lss (321,776 bytes)

TMSWhite 2011-07-24 07:29 reporter ~15847	This now works in revision 10579, with versions of tailoring and relevance calculations using much-improved in Javascript.syntax highlighting and tool-tipping. It also supports real-time changes to question visibility and micro-tailoring on the current page without visibility.

TMSWhite 2011-10-28 23:03 reporter ~16565	This is available in version 2.0alpha+

Date Modified	Username	Field	Change
2011-04-11 20:37	TMSWhite	New Issue
2011-04-19 20:36	TMSWhite	Note Added: 14849
2011-05-07 11:58	c_schmitz	Status	new => acknowledged
2011-05-27 13:41	Mazi	Assigned To	=> c_schmitz
2011-05-27 13:41	Mazi	Status	acknowledged => assigned
2011-05-27 13:42	Mazi	Note Added: 15135
2011-05-27 15:53	TMSWhite	Note Added: 15136
2011-05-27 20:21	TMSWhite	Note Added: 15138
2011-06-10 00:24	TMSWhite	Note Added: 15393
2011-06-10 08:27	TMSWhite	Note Added: 15395
2011-06-10 08:28	TMSWhite	File Added: limesurvey_survey_showing_conditional_tailoring_using_equation_parser.lss
2011-06-10 08:31	TMSWhite	Note Added: 15396
2011-06-10 09:06	TMSWhite	Note Added: 15398
2011-06-10 09:15	TMSWhite	File Added: issue_05103.patch
2011-06-10 09:20	TMSWhite	Note Added: 15399
2011-06-11 21:49	TMSWhite	Note Added: 15411
2011-06-12 09:38	TMSWhite	File Added: issue_05103-rev2.patch
2011-06-12 09:40	TMSWhite	Note Added: 15412
2011-06-12 09:44	TMSWhite	Note Edited: 15412	View Revisions
2011-06-12 09:44	TMSWhite	File Added: limesurvey_survey_showing_conditional_tailoring_using_equation_parser-rev2.lss
2011-06-13 18:45	TMSWhite	Note Added: 15419
2011-06-16 08:57	TMSWhite	File Added: issue_05103-20110616.patch
2011-06-16 08:58	TMSWhite	File Added: ExpressionManager-plus-all-question-types.lss
2011-06-16 09:03	TMSWhite	Note Added: 15451
2011-06-16 09:45	TMSWhite	File Added: issue_05103-20110616a.patch
2011-06-16 09:46	TMSWhite	Note Edited: 15451	View Revisions
2011-06-16 14:12	TMSWhite	Note Added: 15455
2011-06-16 14:12	TMSWhite	File Added: ExpressionMaanger-Screenshots.zip
2011-06-16 14:12	TMSWhite	File Added: ExpressionManager-source.zip
2011-06-17 08:28	TMSWhite	Note Added: 15472
2011-06-17 12:00	TMSWhite	File Added: issue_05268-20110617.patch
2011-06-17 12:01	TMSWhite	Note Added: 15474
2011-06-17 12:29	TMSWhite	Note Added: 15476
2011-06-18 00:32	TMSWhite	Note Added: 15483
2011-06-18 00:35	TMSWhite	File Added: issue_05268-20110617-v2.patch
2011-06-19 18:11	TMSWhite	Note Added: 15509
2011-06-19 18:59	TMSWhite	Relationship added	related to 05288
2011-06-19 19:01	TMSWhite	Relationship added	related to 05269
2011-06-19 19:02	TMSWhite	Relationship added	related to 05268
2011-06-19 19:03	TMSWhite	Relationship added	related to 05104
2011-06-19 19:03	TMSWhite	Relationship added	related to 05279
2011-07-02 07:19	TMSWhite	Note Added: 15614
2011-07-04 09:36	TMSWhite	Note Added: 15626
2011-07-11 23:22	TMSWhite	Note Added: 15731
2011-07-11 23:25	TMSWhite	File Added: RegressionTester-en-de.lss
2011-07-12 03:58	TMSWhite	Relationship added	related to 04424
2011-07-24 07:29	TMSWhite	Note Added: 15847
2011-10-28 23:03	TMSWhite	Note Added: 16565
2011-10-28 23:03	TMSWhite	Status	assigned => resolved
2011-10-28 23:03	TMSWhite	Fixed in Version	=> 2.00
2011-10-28 23:03	TMSWhite	Resolution	open => fixed
2011-10-28 23:03	TMSWhite	Assigned To	c_schmitz => TMSWhite
2012-06-21 13:22	c_schmitz	Status	resolved => closed

View Issue Details

Relationships

Activities

Issue History