View Issue Details

IDProjectCategoryView StatusLast Update
08040Development Import/Exportpublic2014-01-11 09:04
ReportermfaberAssigned Tomfaber 
PrioritynormalSeverityfeature 
Status closedResolutionfixed 
Product Version 
Target VersionFixed in Version2.05 
Summary08040: implement STATA export functionality
Description

STATA is a statistical software package. While data from LS can be imported via Excel or CSV, the number of variables will be limited to <255 (Excel) and value labels will be missing (Excel AND CSV method).

My plan is to add STATA-XML export functionality to LS.

How I plan to implement:
Expand the "export results to application" screen (after CSV/EXCEL/WORD/PDF/HTML a sixth option "STATA-XML"). This will be a "one-file solution". No separate data and command files necessary like for SPSS & R.

Files planned to change: exportresults_view, exportresults_helper and a new stataxmlwriter.php.

If this works ok...I'd like to see if it's feasible replacing or complementing the old xls-format with xlsx-format export.

TagsNo tags attached.

Relationships

related to 06651 closedmfaber User patches Export to Stata functionality for LS2.0+ 

Activities

mdekker

mdekker

2013-07-23 16:16

reporter   ~25854

Please create this as a plugin in the 2.05 branch. I added a plugin for r-export to 2.05 already but did not do the interface for it yet. It can only be used via remote control at the moment.

mfaber

mfaber

2013-07-23 16:31

reporter   ~25855

Oh ok, I already wondered what this plugin does ;). Will have a look at the plugin architecture then.

mfaber

mfaber

2013-07-23 17:55

reporter   ~25856

mdekker: I have two questions on the plugin system:

1) where will the plugin "plug in"? On a yet-to-be-coded generic plugin page or on an existing page (export_view) or ??

2) how do you call a plugin through remotecontrol? I cannot find a remotecontrol function that would allow me to access plugins. (Neither in the wiki nor on the status page of /limesurvey/index.php/admin/remotecontrol).

mdekker

mdekker

2013-07-24 09:26

reporter   ~25862

1) On the existing page

I have some trouble to decide on how to handle extra selectors / disabled selectors and extra info (like spss/r howto). I will port all exports to a plugin so the system admin can decide what exports he wants to use.

2) Using remote control there is no interface needed to select a different kind of export and since there is no validation (yet), you can just use rsyntax as export style and you will get the plugin rsyntax if it is available.

I implemented that as a quickfix for needed functionality. I still need to port the spss syntax and allow plugins in the interface so all can be cleaned up. If you want to have a go at the interface you are more then welcome. If you want to focus on this new export first, that is ok with me too. You are welcome to share your ideas on the export interface.

mfaber

mfaber

2013-07-24 18:26

reporter   ~25867

Thanks for the info, mdekker. I will first go for the new export method because that's what's needed here.

For the interface I could imagine a radio button (to choose the export plugin) and an options button next to each radio button (if necessary).
Most times, users just need to choose a plugin and hit "Export", no options necessary.

I do see your point with the R/SPSS methods (2 files and also some ad-hoc help needed). Why not solve it by packing both file types (CSV and command file) together with a "howto.txt" file into a zip that is downloaded to the client. This would IMHO also be more convenient for the user.

mfaber

mfaber

2013-07-28 11:03

reporter   ~25879

Hi mdekker. I am probably half there.

I pull the data I need from LS' tables to create a fieldmap and a response map. Now I discovered SurveyDAO.php and I am tempted to rewrite my code so SurveyDAO will be my data basis. Sometimes you have to learn the hard way ;).

However the response part of the DAO-created $survey array is empty or, when I call loadSurveyResults(), there's basically only an SQL query.
May I kindly ask if this is intended behaviour of the function or is the DAO still work in progress?
Thanks,
M

mdekker

mdekker

2013-07-29 09:09

reporter   ~25884

Check out exportresults_helper.php (line 128 and up) that is where the survey object is loaded and populated. Then in the write() method of the Writer it loops through the results around line 400.

The outputRecord method is called for every response it finds, so you don't have to worry about loading the results yourself.

mfaber

mfaber

2013-08-05 13:21

reporter   ~25929

Sorry, forgot to reply...thanks again for the hint. I rewrote some of my code so it now uses the $survey object and the outputRecord function instead of DB-calls to create a custom fieldmap and a response array. Hope to issue a pull request with a stata-xml export this week.

For the export to STATA, i need to do a fair amount of data manipulation before writing the actual xml. For example I need to know what the maximum string length in one variable is to write it in a certain section of the export file.

That means I cannot really use the outputRecord method as intended but first need to collect all answers in an array, count the lengths of strings and then go back to create/modify the fieldmap.
Or, put basically everythin in the close() function, where the outputRecord() is already finished.

I wonder if the writer-part of the plugins should maybe get a different form of access to the responses than response by reponse via outputrecord(). But I might also have overlook somethind and I also see that for CSV-files this is probably the most convenient way to go...

mfaber

mfaber

2013-08-06 00:10

reporter   ~25933

Issued pull request on gitgub: https://github.com/LimeSurvey/LimeSurvey/pull/104

Would be happy if this could find its way into 2.05!

mfaber

mfaber

2013-08-12 09:30

reporter   ~25953

implemented in 2.05

Related Changesets

LimeSurvey: 2.05 94a80521

2013-08-05 21:59:25

mfaber


Committer: mfaber Details Diff
New feature 08040: STATA export functionality

DEV: Implements XML-export for statistical software STATA
DEV: Thanks to mdekker for his support regarding the plugin
DEV: architecture.
Affected Issues
08040
add - application/core/plugins/ExportSTATAxml/ExportSTATAxml.php Diff File
add - application/core/plugins/ExportSTATAxml/STATAxmlWriter.php Diff File
mod - application/views/admin/export/exportresults_view.php Diff File

LimeSurvey: 2.05 0ff722bc

2013-08-08 21:52:34

mfaber


Committer: mfaber Details Diff
Fixed STATA export: No output when debug!=0

DEV: Menno, both reported issues should be fixed with this commit.
DEV: Thanks for the close look!
Affected Issues
08040
mod - application/core/plugins/ExportSTATAxml/ExportSTATAxml.php Diff File
mod - application/core/plugins/ExportSTATAxml/STATAxmlWriter.php Diff File

Issue History

Date Modified Username Field Change
2013-07-23 16:08 mfaber New Issue
2013-07-23 16:10 mfaber Relationship added related to 06651
2013-07-23 16:16 mdekker Note Added: 25854
2013-07-23 16:31 mfaber Note Added: 25855
2013-07-23 17:55 mfaber Note Added: 25856
2013-07-24 09:26 mdekker Note Added: 25862
2013-07-24 18:26 mfaber Note Added: 25867
2013-07-25 03:56 mfaber Assigned To => mfaber
2013-07-25 03:56 mfaber Status new => assigned
2013-07-28 11:03 mfaber Note Added: 25879
2013-07-29 09:09 mdekker Note Added: 25884
2013-08-05 13:21 mfaber Note Added: 25929
2013-08-06 00:10 mfaber Note Added: 25933
2013-08-12 09:26 mfaber Changeset attached => LimeSurvey 2.05 94a80521
2013-08-12 09:30 mfaber Note Added: 25953
2013-08-12 09:30 mfaber Status assigned => resolved
2013-08-12 09:30 mfaber Fixed in Version => 2.05
2013-08-12 09:30 mfaber Resolution open => fixed
2013-08-12 09:30 mfaber Changeset attached => LimeSurvey 2.05 0ff722bc
2014-01-11 09:04 mfaber Status resolved => closed