5

I have a problem with charsets.

I parsed a csv file in google-app-engine and I'm posting to an uiapp table. But I checked special characters like áéíóú and those are not well displayed (?square symbol).

When I was setting up my code I played writing the string imported to a google docs document and it worked the same.

some advice please?

I search for:

  • a global charset definition to the code. or
  • string var transformation that makes the chars appear like I want to. (avoiding html &number definitions.
  • Is this related to the blob object?

The thing is important i come from spain and we need such characters.

app that get's a csv ';' delimited file and shows it's content

I post all my code, it's barely as the tutorial that is given.

function arreglaUrl(cadena){
  var texto = cadena[cadena.length - 2]
  if (texto == ''){
    cadena[cadena.length - 2] = 'Sin enlace';
  }
  else{
    cadena[cadena.length - 2] = '<center><a href=\"'+ texto + '\">Link.</a></center>' ;
  };

}

function parsedCSV(){

  var listaArchivos = DocsList.getFolderById('XXXXX').getFiles()

  for (var i = 0; i < listaArchivos.length; i++) {
    if (listaArchivos[i].getName() == 'baul.csv'){
      var origen = listaArchivos[i];
      };
  } 
  var texto = origen.getContentAsString();
  var arra = Utilities.parseCsv(texto,";");
  return(arra);
}



function doGet() {

  var datos = parsedCSV()


  var baul = Charts.newDataTable()
    for (i = 0; i < datos[0].length; i++){

        baul.addColumn(Charts.ColumnType.STRING, datos[0][i])   
    }

    for (i = 1; i < datos.length; i++){
      arreglaUrl(datos[i]) // this only makes some html i need to post some links
      baul.addRow(datos[i])

     }
    baul.build();



  var sectorFilter = Charts.newCategoryFilter()
      .setFilterColumnLabel("sector")
      .build();

  var tipoFilter = Charts.newCategoryFilter()
      .setFilterColumnLabel("tipo")
      .build();
  var searchFilter = Charts.newStringFilter()
      .setFilterColumnLabel("Titulo")
      .build();
  var searchDesc = Charts.newStringFilter()
      .setFilterColumnLabel("descripcion")
      .build();


  var tableChart = Charts.newTableChart().setOption('allowHtml', true).setDimensions(0,0)
      .build();

    var dashboard = Charts.newDashboardPanel()
      .setDataTable(baul)
      .bind([sectorFilter, tipoFilter, searchFilter, searchDesc], [tableChart])
      .build();

  var uiApp = UiApp.createApplication().setTitle('Baul de Recursos');
  var anchoTotal = '100%';
  dashboard.add(uiApp.createVerticalPanel()

                .add(uiApp.createHorizontalPanel()
                     .add(sectorFilter)
                     .add(tipoFilter)
                     .setSpacing(15)

                    )

                .add(uiApp.createHorizontalPanel()        
                     .add(searchFilter)
                     .add(searchDesc)
                     .setSpacing(15)                
                    )

                    .add(uiApp.createHorizontalPanel()
                         .add(tableChart).setBorderWidth(1).setHorizontalAlignment(UiApp.HorizontalAlignment.CENTER).setWidth(anchoTotal)
                        )

                );

  uiApp.add(dashboard);
  return uiApp;
}
Antonio E.
  • 359
  • 1
  • 9
  • 27
  • changed, there's the code and another link i hope it works – Antonio E. Nov 14 '13 at 21:21
  • Well, in all my experience with Google in general (Google Docs particularly), 'special' characters are not displayed well. For example, if I write a song in French that uses characters like `é, è, î, à`, etc, whatever sub-system Google services use to process to text actually completely removes those characters. I have no idea why, and it's something I wish they would fix. Unicode? I don't know what they're using, but for a lot of services it just plain will *not* render text correctly, and this may be the problem. Though I hope this isn't the case... – Chris Cirefice Nov 14 '13 at 23:04

1 Answers1

4

I found it, we need to get the content of the file first with a Blob object. This function is the one I use to parse some csv info into an array:

function parsedCSV(){
  
 //searching the file. This gets only one file in var origen
  var listaArchivos = DocsList.getFolderById('XXXXXXX').getFiles()
  
  for (var i = 0; i < listaArchivos.length; i++) {
    if (listaArchivos[i].getName() == 'baul.csv'){
      var origen = listaArchivos[i];
  };
}

// HERE IS THE GOOD DEFINITION OF CHAR:
var texto2= origen.getBlob().getDataAsString('ISO-8859-1');
// I put all the corrected text in an array

var arra = Utilities.parseCsv(texto2,";");
return(arra);
}

This is the solved thing: https://script.google.com/macros/s/AKfycbyHa-bLWBHBr3qifbvzxecqGgGUYX8mhyo-TKoyfGvy/exec

The trick:

var textVariableName = fileObjectVariableName.getBlob().getDataAsString('ISO-8859-1');
Community
  • 1
  • 1
Antonio E.
  • 359
  • 1
  • 9
  • 27
  • Fantastic! It's good to know that you can set the character encoding (`ISO-8859-1`) when you have a `Blob` of data. I didn't know that! Can you create a **public copy** of your UI and share it in your answer? Currently you are linking to your (I assume) *original*, and I don't have permission to access it. I would love to see your script in action! – Chris Cirefice Nov 18 '13 at 15:48
  • 1
    @Chris- as already mentioned in this post : http://stackoverflow.com/questions/19756252/google-apps-script-how-to-fix-content-is-not-allowed-in-prolog/19773159#19773159. But it was for another use case :-) – Serge insas Nov 18 '13 at 16:26
  • @Sergeinsas, very very nice. I didn't know about that feature and as a matter of fact it makes encoding and special characters a bit easier to manage in Drive. Now here's an interesting question - why is it that when I download documents that I wrote in French (or uploaded as .docx) to Drive, or print, that accented characters are erased? It seems like Drive uses some obscure encoding and all the docs need to be changed to UTF-8 for them to work properly. Is it just me? I could automate conversions with a script, but there has to be a better way... haha – Chris Cirefice Nov 18 '13 at 16:33
  • I'm surprised you get that issue, I don't and most of my documents are in french... did you check the language setting in the doc itself ? – Serge insas Nov 18 '13 at 16:37
  • I'm not as clever for answer you. I'm only playing with the google apps. – Antonio E. Nov 19 '13 at 13:32
  • Serge: I didn't. I have to explore de google documentation. I don't know if there is a method to change or get the charset of a whole document. – Antonio E. Nov 19 '13 at 13:33
  • @ChrisCirefice: the link is updated i think you should look the searchable table with some test data, you should see the special characters correctly – Antonio E. Nov 19 '13 at 14:44
  • 1
    @Sergeinsas I didn't know Docs had a language setting. I just assumed that it would keep that information when I uploaded it in .docx, where Word had the language set to French. Drive for the iPad doesn't have a language option, which may be the source of the problem because that's where I do most of my writing! And user2209264: very nice! I am happy to see that it's working properly, and this is a neat tool - I think I will be building something like this in the future for my work to keep track of our film collection :) – Chris Cirefice Nov 19 '13 at 18:12
  • @Sergeinsas **I think the language setting is about the strings and text variable types, some of you have said you didn't have the problem with French characters.** The idea of this tool, is that someone offline updates a csv file and then replaces it in Drive. I wanted to do like this (instead of using a spreadsheet form) because i have some no-geek partners that wont know how to use the form, or logging into google drive open to correct the spreadsheet if something is wrong, some of them doesn't have google acount... – Antonio E. Nov 28 '13 at 17:30