|
|
as I said on the forum, it seems to me that what is returned might not be JSON at all - just a 500 error page:
original function is:
get_session_key <- function() {
body.json = list(method = "get_session_key",
id = 1,
params = list(username = getOption('lime_username'),
password = getOption('lime_password')))
r <- POST(getOption('lime_api'),
content_type_json(),
body = jsonlite::toJSON(body.json,
auto_unbox = TRUE))
session_key <- as.character(jsonlite::fromJSON(content(r, encoding="utf-8"))$result)
session_cache$session_key <- session_key
return(session_key)
}
If you execute it by parts:
prepare POST request
body.json = list(method = "get_session_key",
id = 1,
params = list(username = getOption('lime_username')))
that is OK and further gets a response from LS
r <- POST(getOption('lime_api'),
content_type_json(),
body = jsonlite::toJSON(body.json,
auto_unbox = TRUE))
error then arises in the next step:
session_key <- as.character(jsonlite::fromJSON(content(r, encoding="utf-8"))$result)
Error: Argument 'txt' must be a JSON string, URL or file
if you look into what content of r is (btw it is library httr::content() function) you get html document:
content(r, encoding="utf-8")
{html_document}
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
[1] <head>\n<meta http-equiv="Content-Type" content="text/html; charset=utf-8">\n<title>ArgumentCoun ...
[2] <body>\n<div class="container">\n\t<h1>ArgumentCountError</h1>\n\n\t<p class="message">\n\t\tToo ...
if you look at the structure of r (seeing whole response):
str(r)
List of 10
$ url : chr "https://lime.survey.server.org/index.php/admin/remotecontrol"
$ status_code: int 500
$ headers :List of 17
..$ connection : chr "Keep-Alive"
..$ keep-alive : chr "timeout=5, max=100"
..$ x-powered-by : chr "PHP/7.4.33"
..$ expires : chr "Thu, 19 Nov 1981 08:52:00 GMT"
..$ cache-control : chr "no-store, no-cache, must-revalidate"
..$ pragma : chr "no-cache"
..$ content-type : chr "text/html; charset=UTF-8"
..$ transfer-encoding : chr "chunked"
..$ content-encoding : chr "gzip"
..$ vary : chr "Accept-Encoding"
..$ date : chr "Mon, 20 Nov 2023 20:34:41 GMT"
..$ server : chr "LiteSpeed"
..$ strict-transport-security: chr "max-age=63072000; includeSubDomains"
..$ x-frame-options : chr "SAMEORIGIN"
..$ x-content-type-options : chr "nosniff"
..$ x-xss-protection : chr "1; mode=block"
..$ alt-svc : chr "h3=\":443\"; ma=2592000, h3-29=\":443\"; ma=2592000, h3-Q050=\":443\"; ma=2592000, h3-Q046=\":443\"; ma=2592000"| truncated
..- attr(, "class")= chr [1:2] "insensitive" "list"
$ all_headers:List of 1
..$ :List of 3
.. ..$ status : int 500
.. ..$ version: chr "HTTP/1.1"
.. ..$ headers:List of 17
.. .. ..$ connection : chr "Keep-Alive"
.. .. ..$ keep-alive : chr "timeout=5, max=100"
.. .. ..$ x-powered-by : chr "PHP/7.4.33"
.. .. ..$ expires : chr "Thu, 19 Nov 1981 08:52:00 GMT"
.. .. ..$ cache-control : chr "no-store, no-cache, must-revalidate"
.. .. ..$ pragma : chr "no-cache"
.. .. ..$ content-type : chr "text/html; charset=UTF-8"
.. .. ..$ transfer-encoding : chr "chunked"
.. .. ..$ content-encoding : chr "gzip"
.. .. ..$ vary : chr "Accept-Encoding"
.. .. ..$ date : chr "Mon, 20 Nov 2023 20:34:41 GMT"
.. .. ..$ server : chr "LiteSpeed"
.. .. ..$ strict-transport-security: chr "max-age=63072000; includeSubDomains"
.. .. ..$ x-frame-options : chr "SAMEORIGIN"
.. .. ..$ x-content-type-options : chr "nosniff"
.. .. ..$ x-xss-protection : chr "1; mode=block"
.. .. ..$ alt-svc : chr "h3=\":443\"; ma=2592000, h3-29=\":443\"; ma=2592000, h3-Q050=\":443\"; ma=2592000, h3-Q046=\":443\"; ma=2592000"| truncated
.. .. ..- attr(, "class")= chr [1:2] "insensitive" "list"
$ cookies :'data.frame': 1 obs. of 7 variables:
..$ domain : chr "#HttpOnly_lime.survey.server.org"
..$ flag : logi FALSE
..$ path : chr "/"
..$ secure : logi TRUE
..$ expiration: POSIXct[1:1], format: "Inf"
..$ name : chr "PHPSESSID"
..$ value : chr "ffd7045b984f468749a8cb1ba5ca9500"
$ content : raw [1:20798] 3c 21 44 4f ...
$ date : POSIXct[1:1], format: "2023-11-20 20:34:41"
$ times : Named num [1:6] 0 0.00188 0.16919 0.32191 1.57809 ...
..- attr(, "names")= chr [1:6] "redirect" "namelookup" "connect" "pretransfer" ...
$ request :List of 7
..$ method : chr "POST"
..$ url : chr "https://lime.survey.server.org/index.php/admin/remotecontrol"
..$ headers : Named chr [1:2] "application/json, text/xml, application/xml, /" "application/json"
.. ..- attr(, "names")= chr [1:2] "Accept" "Content-Type"
..$ fields : NULL
..$ options :List of 4
.. ..$ useragent : chr "libcurl/7.84.0 r-curl/5.0.1 httr/1.4.6"
.. ..$ post : logi TRUE
.. ..$ postfieldsize: int 65
.. ..$ postfields : raw [1:65] 7b 22 6d 65 ...
..$ auth_token: NULL
..$ output : list()
.. ..- attr(, "class")= chr [1:2] "write_memory" "write_function"
..- attr(, "class")= chr "request"
$ handle :Class 'curl_handle' <externalptr>
- attr(*, "class")= chr "response"
|
|
|
|
I have just spend quite a while installing r and testing your issue.
First problem i have is that your code makes no sense to me. Why are you sending a json request and then trying to decode XML? There are two RPC: json and xml. For json you send json, get back json. For xml you send xml, get back xml. Why are you trying to mix them?
Second problem I have is that your code does not produce the result you say. I don't get any 500 error. I only see:
rvest::html_text(xml)
Error in UseMethod("xml_text") :
no applicable method for 'xml_text' applied to an object of class "list"
which makes sense because you try to decode json as xml
my complete code:
Load library
library(limer)
library(httr)
library(rvest)
Setup API details
options(lime_api = 'http://limesurvey-master-8_1/admin/remotecontrol')
options(lime_username = 'admin')
options(lime_password = 'admin')
body.json = list(method = "get_session_key",
id = 1,
params = list(username = getOption('lime_username'), password = getOption('lime_password')))
r <- POST(getOption('lime_api'), content_type_json(), body = jsonlite::toJSON(body.json, auto_unbox = TRUE))
xml <- content(r, encoding="utf-8")
print(xml)
rvest::html_text(xml)
I don't use R much, so maybe I made a mistake, but I don't see your error. I just see correct json response with a valid session key. |
|
|
|
your steps to reproduce has a 500 error
but your note has an r error
with my testing I can only reproduce the r error
I see no limesurvey error
and the r error is because you do not properly parse json |
|
|
|
Sorry, I realize I should have clarified that the 2 lines of code I signposted as “to check error message” may be faulty due to my inexperience with JSON.
The R / liner error that is the actual problem is described earlier in the forum post: https://forums.limesurvey.org/forum/installation-a-update-issues/127953-remotecontrol-doesn-t-work-after-update-to-php-8-1?start=12#252424
The error occurs when wanting to get the session key as per limer documentation:
Load library
library(limer)
Setup API details
options(lime_api = 'http://example.com/limesurvey/admin/remotecontrol')
options(lime_username = 'put_username_here')
options(lime_password = 'put_password_here')
Do stuff with LimeSurvey API
get_session_key() # Log in
Error:
Argument 'txt' must be a JSON string, URL or file |
|
|
|
Thanks for looking into this mfavetti.
I think we may be getting confused by rvest and xml. The original code in getting session key is quite short and simple and relies on httr only (to make POST request)
see: https://github.com/cloudyr/limer/blob/master/R/get_session_key.R
The actual function is pretty short
get_session_key <- function(username = getOption('lime_username'),
password = getOption('lime_password')) {
body.json = list(method = "get_session_key",
id = " ",
params = list(username = username,
password = password))
r <- POST(getOption('lime_api'), content_type_json(),
body = jsonlite::toJSON(body.json, auto_unbox = TRUE))
session_key <- as.character(jsonlite::fromJSON(content(r, encoding="utf-8"))$result)
session_cache$session_key <- session_key
session_key
}
You can play it line by line and get the result I described above.
jsonlite library cannot parse r that was returned
r should have been json
as.character(jsonlite::fromJSON(content(r, encoding="utf-8"))$result)
if you use jsonlite::validate on r you get error. So something seems to be wrong with what is returned. There was a recent change that iceclimber referred to (https://bugs.limesurvey.org/view.php?id=19214) so it seems what is returned has changed.
> jsonlite::validate(httr::content(r, encoding = "utf-8"))
Error in jsonlite::validate(httr::content(r, encoding = "utf-8")) :
is.character(txt) is not TRUE
also if you make sure to convert r to character string you get non-json structure
> jsonlite::validate(as.character(httr::content(r, encoding = "utf-8")))
[1] FALSE
attr(,"offset")
[1] 1
attr(,"err")
[1] "lexical error: invalid char in json text.\n <!DOCTYPE html PUBLIC \"-//W3C//\n (right here) ------^\n"
|
|
|
|
idk what the difficulty is...
library(limer)
library(httr)
library(jsonlite)
options(lime_api = 'http://limesurvey-master-8_1/admin/remotecontrol')
options(lime_username = 'admin')
options(lime_password = 'admin')
body.json = list(
method = "get_session_key",
id = 1,
params = list(
username = getOption('lime_username'),
password = getOption('lime_password')
)
)
r <- POST(
getOption('lime_api'),
content_type_json(),
body = jsonlite::toJSON(
body.json,
auto_unbox = TRUE
)
)
body <- content(r, "text", encoding="utf-8")
json <- jsonlite::fromJSON(body)
session_key <- as.character(json$result)
print(session_key)
yields
[1] "NPz1zF_pcsil250gSHXFLWDbOdvG670Y" |
|
|
|
i don't think its a problem with limesurvey
i don't think its a problem with limer
i think its an issue with your r code parsing a json response |
|
|
|
oh i see the error if you use limer get_session_key() directly |
|
mfavetti
2023-11-21 11:59
developer
~78575
Last edited: 2023-11-21 12:10
|
ok the problem is in limer
if the response content type is application/json, httr automagically handles the conversion from json
limer explicitly does the conversion (because limesurvey used to return a weird content type) so it's doing it twice now
limer needs to update to remove jsonlite::fromJSON call |
|
mfavetti
2023-11-21 12:01
developer
~78576
Last edited: 2023-11-21 12:11
|
or could specify type=text and continue manual json parsing |
|
mfavetti
2023-11-21 12:18
developer
~78580
Last edited: 2023-11-21 12:21
|
i made a pr for limer to fix the issue for their redundant parsing, I think we can close this issue as it's not a limesurvey bug
https://github.com/cloudyr/limer/pull/66 |
|
|
|
Amazing, thank you mfavetti for finding the solution so quickly! |
|
|
|
@mfavetti Are you sure this does not break for any Limesurvey before 6.3.5? See my remark on github / limer |
|
|
|
yeah I saw, I don't think it does
before limesurvey had weird content type, so httr didn't do anything and limer parsed json
now limesurvey has standard content type, so httr knows to parse json and limer parses json a second time, which causes error
my pr just makes limer parse json of the response as text (so not parsed already by httr) which should handle both cases |
|
|
|
Thanks, mfavetti, very much for looking into it. The code we were posting is from limer which we both independently borrowed from original limer (https://github.com/cloudyr/limer/blob/master/R/get_session_key.R). Our own script R code is not here and does not even try to parse the json response. But I think you are right, the code that you posted does work. The obvious difference is here:
Your code:
body.json = list(method = "get_session_key",
id = 1,
params = list(username = getOption('lime_username'),
password = getOption('lime_password')))
The official limer code:
body.json = list(method = "get_session_key",
id = " ",
params = list(username = username,
password = password))
But that really should not change anything, unless I am mistaken. You have other slight differences in the code and if I make a custom function my_get_session_key() utilising your code, I can get the session key. However if I run get_session_key() from the installed limer library I get the same error Error: Argument 'txt' must be a JSON string, URL or file. |
|
|
|
OK, great, thanks @mfavetti for identifying the problem! Makes absolute sense and I did not spot the double conversion yet. |
|
|
|
I created a new fork with the PR by @mfavetti
You can test it by running
devtools::install_github("Jan-E/limer", force = TRUE)
See https://github.com/Jan-E/limer |
|
|
|
Thank you, @JanE and @mfavetti.
I tested @JanE's fork with LS 6.3.4, 6.3.5 and 6.3.6. I confirm it works for all versions!
To be sure, I also tested all it across all three LS versions using different PHP versions: 7.4.33; 8.1.22; and 8.2.9.
It works as expected.
Here is my R code:
devtools::install_github("Jan-E/limer", force = TRUE)
library(limer)
options(lime_api = 'my_remotecontrol_url')
options(lime_username = 'my_username')
options(lime_password = 'my_password')
session_key <- get_session_key() # Log in
release_session_key() # Log out |
|
|
|
Thanks everyone, I also confirm it works. PHP 7.4, LS CE 6.3.6. |
|
|
|
Updating to 5.6.45+ had the same effect, because the change to the application/json header has been backported. See the release notes: https://github.com/LimeSurvey/LimeSurvey/commit/a7f35a5c57cc8342c22173198a8dd42ef2cd550a |
|
