View Issue Details

IDProjectCategoryView StatusLast Update
0006131phpList 3 applicationSubscriber Importpublic21-06-18 13:05
Reportershumisha 
PrioritynormalSeverityblockReproducibilityalways
Status resolvedResolutionwon't fix 
Product Version2.10.2 
Target Versionnext majorFixed in Version 
Summary0006131: Import suscriber from csv do not work
DescriptionHello !

I have tried importing email lists and customer data using csv import, but it was not working either using 'Import with same attributes' or using 'Import wiht different attributes'. So I have looked into it and tried to make some changes to import1.php and importcsv.php. There were a number of small errors in both files, and I hope I have them running well now.
The main change has to do with csv parsing. Prior to this change, imported csv were not supposed to have any , : < or > or other special characters. There was a test at the beginning of import1.php to control this, and many people would get an 'illegal characters' errors. I have removed the test and added a parsing function (from php.net) able to import a properly formated csv file into php variable :
    
 // $uservalues = explode($import_field_delimiter,$line); REMOVED
    $uservalues = csv_string_to_array($line); // ADDED

I have also made an Excel macro that make a properly formated csv file from an excel spreadsheet : every date is enclosed in double-quote, and a comma is used as a separator.
There is only one thing left : if there are commas in the data fields, they are properly imported by the csv_string_to_array() function (I checked that), however phplist strips them out at a later stage which I have not been able to spot.

I have posted this on the forum, at http://www.phplist.com/forums/viewtopic.php?t=5085
There are more details there. I just thought I would post a report here as well
I have attached fixed importcsv.php and import1.php. Just search for 'shumisha' to find all changes I have made.

TagsNo tags attached.

Relationships

related to 0002705 closed PHPList v2.11 release 

Activities

17-04-06 12:25

 

import.zip (13,380 bytes)

shumisha

21-04-06 08:32

reporter   ~0012908

Sorry, there is a typo in the title, I don't know how to change it. It should read : Import from CSV do not work !

michiel

21-04-06 14:06

manager   ~0012925

thanks, that's a great contribution!

shumisha

21-04-06 14:22

reporter   ~0012928

Thanks Michiel,

Please keep in mind that this needs cleaning up !
Also there is a remaining issue about the commas being stripped out of the data after being imported.

Great software anyway, working fine for me aside from the import issues !

michiel

27-04-06 21:30

manager   ~0013153

Do you mean the function listed here?

http://ar2.php.net/manual/en/function.fgetcsv.php

It doesn't use TABs as delimiters. One of the reasons I like TAB delimited files is that CSV files are unlikely to have a TAB in the data, so it always splits correctly. I always use TAB delimited files and import always works great.

I won't include it in the next bugfix round, but keep the idea for new development issues.

shumisha

28-04-06 12:22

reporter   ~0013193

Hello !
Well, I was not directly referring to this function. I looked it up and found it too difficult to modify phplist to use it. SO instead I used the function csv_string_to_array($str){, that you can find on the same page, just below fgetcsv description, by an anonymous poster. This function takes a string and do the parsing job very well. Indeed it works perfectly, even if there are some commas in between double-quotes (that is embedded within the data). The only problem I have left is that upon importing data using this csv_string_to_array(), commas in between double quotes is properly processed, and stays in the data. However, somewhere at a later stage, phplists strips out these commas, and I just could not find where. Or maybe it was too late in the night !
This is an issue for me, because many times land address will have commas.
Regarding Tab vs commas, I'd like to comment that :
- commas delimited files are very common, many data source can produce them
- one can set the field delimiter used by fgetcsv() as a parameter, so I think it fine to use it even on a TAB delimited file !