0

i need to replcae all multibyte in a regular file.

Sample data

1792834104797 Impression Clickstream Page FPTI
1468294723126 257678 10692349
5c7ee584-6f83-4062-b324-2219922e8848 mobile:msdk:preconnect:: mobile:msdk:preconnect:::Android:live:
7（内测工程机a511） Android 5.1 2.9.10 1792834104797
session_start_gmt=1468294723126&session_start_page=mobile:msdk:preconnect:::Android:live:&session_end_gmt=1468294723126&session_end_page=mobile:msdk:preconnect:::Android:live:&mobile_carrier=CMCC&mobile_app=优步 - Uber|2.9.10|com.ubercab&event_type=im&data_source=msdk&session_length_sec=0.0&user_logged_in=out&eap_source_path=hdfs://stampy/sys/dt/fpti/v1/seq/enrichedevents/2016/07/11/20/part-m-00017&user_guid=5c7ee584-6f83-4062-b324-2219922e8848&client_id=AdtlNBDhgmQWi2xk6edqJVKklPFyDWxtyKuXuyVT-OgdnnKpAVsbKHgvqHHP&geo_city=Wenzhou&session_id_cookie_id=10692349-5c7ee584-6f83-4062-b324-2219922e8848&bot_flag_yn=N&epoch_timestamp_client=1468294684561&server_businessname=mobile&unique_message_id=dd2f76361550a4a9c5b18125feef485d&page_version=Android:live:&device_id=5c7ee584-6f83-4062-b324-2219922e8848&business_channel_name=msdk&user_session_guid=10692349&rosetta_language=zh_CN&geo_state=Zhejiang&ip_address=36.251.188.153&geo_region=Zhejiang&gmt_offset_minutes=480&event_time=1468294723126&geo_cntry=CN&geo_country=CN&Web_Session_bot_flag_yn=N&Web_Session_session_mobile_app_version=2.9.10&Web_Session_session_mobile_device=7（内测工程机a511）&Web_Session_session_mobile_os=Android 5.1 1807172862537 session_start_gmt=1468715369910&session_start_page=mobile:consapp:apptransition::didfinishlaunching:android::&session_end_gmt=1468716146342&session_end_page=mobile:consapp:apptransition::willenterbackground:android::&mobile_carrier=Sprint&client_os=Android&is_bot=false&device_type=Mobile Phone&event_type=im&session_length_sec=0.0&prospect_type=0&device_name=Sprint Galaxy S6 (SM-G920P)&eap_source_path=hdfs://stampy/sys/dt/fpti/v1/seq/enrichedevents/2016/07/16/17/part-m-00026&user_guid=7397ebff63b98126&geo_city=Seekonk&geo_zip=02771&device_timestamp=1468715590595&session_id_cookie_id=59906919-7397ebff63b98126&encrypted_customer_id=J4AN28RAWDY46&bot_flag_yn=N&epoch_timestamp_client=1468715590595&mobile_device_id=7397ebff63b98126&unique_message_id=f645670c1550a4a9e9f67ceefc230a85&user_session_guid=59906919&search_keyword= &rosetta_language=en_US&geo_state=NULL&ip_address=66.87.124.151&browser_type=Android Browser&geo_region=MA&gmt_offset_minutes=-240&event_time=1468715591436&geo_cntry=US&geo_country=US&Web_Session_bot_flag_yn=N&Web_Session_session_mobile_app_version=6.4.2&Web_Session_session_mobile_device=SM-G920P&Web_Session_session_mobile_os=Android 6.0.1

i tried to replace like below, But no luck. Can anyone please help 

LANG=C sed 's/[\x80-\xFF]//g'  filename

sed 's/^M//g' filename

sed 's/'"$(printf '\015')"'//g' filename

Thanks in advance!

GopiGowtham
  • 61
  • 1
  • 1
  • 9
  • 1
    Possible duplicate of [Remove non-ascii characters from csv](http://stackoverflow.com/questions/3337936/remove-non-ascii-characters-from-csv) – John Zwinck Jul 17 '16 at 21:30
  • `^M` is normally considered a single character, i.e. Ctrl-M. If you have entered it correctly in your script, `sed` should be able to process it. (Don't use 2 chars, i.e. `^` and `M`, you'll have to find out how your editor generates control chars, with `vi` use the Ctrl-V before to escape pressing the `Enter` key) I can't speak to the other parts of your Q. Good luck. – shellter Jul 17 '16 at 21:51
  • Does this answer your question? [removing multibyte characters from a file using sed](https://stackoverflow.com/questions/3521106/removing-multibyte-characters-from-a-file-using-sed) – Digvijay S Jun 12 '20 at 13:45

0 Answers0