We Keep Coding

Which romanization standard should be used to improve ICU4j transliteration for Arabic-Latin?

We have a requirement to transliterate Arabic text to Latin characters(without diacritical marks) and display them to users.
We are currently using IBM ICU4j for this.
The API doesn't trasliterate well the Arabic text into proper readable latin characters. Refer the below examples:
Example
Arabic text :
صدام حسين التكريتي
Google's transliteration output
: Sadaam Hussein al-tikriti
ICU4J's transliteration outuput
: ṣdạm ḥsyn ạltkryty
How can we improve the transliterated output of ICU4j library?
ICU4J gives us an option to write our own rules but we are currently stuck as no one from our team knows Arabic and are unable to find any proper standard that can be followed.

It's took 4 hours me to research out any other source to tackle out this problem.Later i tried ICU4J and find the solution for your problem .You can run the code and see the point which you was missing.
package com.webom.crypt;
import org.apache.commons.lang3.StringEscapeUtils;
import com.ibm.icu.text.Transliterator;
public class Test {
public static String ARABIC_TO_LATIN = "Arabic-Latin";
public static String ARABIC_TO_LATIN_NO_ACCENTS = "Arabic-Latin; nfd; [:nonspacing mark:] remove; nfc";
public static void main(String[] args) {
String ARABICString = "صدام حسين التكريتي";
String unicodeCodes = StringEscapeUtils.escapeJava(ARABICString);
System.out.println("Unicode codes:" + unicodeCodes);
///YOUR WAY
Transliterator ARABICToLatinTrans = Transliterator.getInstance(ARABIC_TO_LATIN);
String result1 = ARABICToLatinTrans.transliterate(ARABICString);
System.out.println("ARABIC to Latin:" + result1);
//MINE WAY
Transliterator ARABICToLatinNoAccentsTrans = Transliterator.getInstance(ARABIC_TO_LATIN_NO_ACCENTS);
String result2 = ARABICToLatinNoAccentsTrans.transliterate(ARABICString);
System.out.println("ARABIC to Latin (no accents):" + result2);
}
}
Just checkout the answer and verify on your own.As the output you receive will be exactly as shown below.
Unicode codes:\u0635\u062F\u0627\u0645 \u062D\u0633\u064A\u0646\u0627\u0644\u062A\u0643\u0631\u064A\u062A\u064A
ARABIC to Latin:ṣdạm ḥsyn ạltkryty
ARABIC to Latin (no accents):sdam hsyn altkryty

how to remove Arabic hashtags?

I am polling tweets from twitter using Twitter4j
and I am trying to filter hashtags from it after I take text from it
I turn it into strings now
I have this String: "892698363371638784:RT #hikids_ksa: اللعبة خطيرة مرا ويبي لها مخ و تفكير و مهارة👌🏻💡
متوفرة في #متجر_هاي_كيدز_الالكتروني .."
I want to remove متجر_هاي_كيدز_الالكتروني as it has Hashtag after it using java
the problem my code didn't work on this input:
"#kaskasomar هيدا بلا مخ متل متل غيرو بيخون الشعب اللبناني وبيتهمو بالارهاب بس لان رأيو بيختلف عن رأي الاخرين #سخيف"
the part سخيف wasn't removed for some reason
this is my method
static String removeHashtags(String in)
{
in = in.replaceAll("#[A-Za-z]+","");//remove English hashtags
in = in.replaceAll("[أ-ي]#+","");//remove Arabic hashtags that have # before it
return in = in.replaceAll("#[أ-ي]+","");//remove Arabic hashtags that have # after it
}

If you're just trying to remove all hash tags in any language, you can write
in = in.replaceAll("#\\p{IsAlphabetic}+", "");
If you specifically want to remove Arabic hash tags, you can write
in = in.replaceAll("#\\p{IsArabic}+", "");
so you don't have to worry about building a regular expression with left-to-right and right-to-left parts. This improves the readability of your code.

The problem is that, in the second line, the + is applied to the hashtag, not the Arabic characters. Fixed version:
in = in.replaceAll("[أ-ي]+#","");

The code below removes Arabic hashtags mixed with English characters and [0-9] using the Unicode
import re
text = "#مرحبا"
reg = r"#[0-9\u0621-\u063A\u0640-\u066C\u0671-\u0674a-zA-Z_]+"
text = re.sub(reg, " ", text)
Example: regexr.com/691cn

Not able to get correct Currency symbol for countries

I tried with java.text.NumberFormat.For denmark english, it gave me 'DKK' .But i need 'kr'.
Locale denmark = new Locale("en","DK");
NumberFormat denmarkFormat = NumberFormat.getCurrencyInstance(denmark);
System.out.println("Denmark: " + denmarkFormat.format(num));
I tried with java.util.Currency.getSymbol(). For denmark english,it also gave me 'DKK' only.
Locale denmark = new Locale("en","DK");
Currency curr = Currency.getInstance(denmark);
String symbol = curr.getSymbol(denmark);
(OR)
I can get all my custom currency symbols in jsp but if i pass that symbol to java,it is not able to get it properly.For UK,'£' symbol passed from jsp is received as £ in java. This is how i get it in java,
String Currency=(String)pageContext.getRequest().getAttribute("CurrencySymbol");
Getting solution for anyone of these problems will be great.Thanks in advance.
I need the below currency symbols,
USA-$
CA-$
CN-¥
PL-zł
TR-TRY
NL-EUR
SE-kr
DK-kr
FI-EUR
AU-$
DE-EUR
CH-CHF
GB-£
IT-EUR
AT-EUR
FR-EUR
ES-EUR
BE-EUR
IE-EUR
PT-EUR
NO-kr
HK-$

Android Localization query

I have been working on localisation for my app and cant seem to find any information about how to handle decimal values and dates from different locals to store in sqllite.
for example:
German decimal 123,53
Uk decimal 123.53
So how do you convert from an edittext field to a valid decimal. At the moment I have my code outputting to a textview rather than sql just for testing. The below code works great when using UK decimal but if I use the german decimal it fails!!
Configuration sysConfig = getResources().getConfiguration();
Locale curLocale = sysConfig.locale;
NumberFormat nf = NumberFormat.getInstance(curLocale);
String convertedString = nf.format(Double.parseDouble(EditTextField.getText().toString()));
TextView showLocalisedNumeric;
showLocalisedNumeric = (TextView)findViewById(R.id.TestNumericValue);
showLocalisedNumeric.setText(convertedString);
I have not started with dates yet but I am assuming converting dates for is more straight forward.

After some time working on this I found the solution for localisation, taking a value from input and parsing it to a format that can be understood by SQLlite - For this exercise and to reduce code I have just set it to output to a text view.
// set initial value to variables
String convertedString = "";
double parsedValue = 0.0;
//get value from text field
EditText EditTextField =(EditText)findViewById(R.id.TestNumericValueEntered);
String valueFromInput = EditTextField.getText().toString();
//Get current Locale from system
Configuration sysConfig = getResources().getConfiguration();
Locale curLocale = sysConfig.locale;
//Set number formats
NumberFormat nf_in = NumberFormat.getNumberInstance(curLocale);
NumberFormat nf_out = NumberFormat.getNumberInstance(Locale.UK);
//try to parse value, otherwise return error message
try {
parsedValue = nf_in.parse(valueFromInput).doubleValue();
// use nf_out.setMaximumFractionDigits(3) to set max number of digits allowed after decimal;
convertedString = nf_out.format(parsedValue);
}catch(ParseException e){
convertedString = "Unable to translate value";
}
//Output result
TextView showLocalisedNumeric;
showLocalisedNumeric = (TextView)findViewById(R.id.TestNumericValue);
showLocalisedNumeric.setText(convertedString);
As I was adding this answer I realised that a nice addition to the code would be to check if the current locale is the same as the one you plan to parse to, if so then the conversion(parsing) can be skipped.

Android : Error SimpleDateFormat Unknown pattern character 'u'

I use java 1.7.25
but found this error. what should I do?
FATAL EXCEPTION: main
java.lang.IllegalArgumentException: Unknown pattern character 'u'
at java.text.SimpleDateFormat.validateFormat(SimpleDateFormat.java:264)
at java.text.SimpleDateFormat.validatePattern(SimpleDateFormat.java:319)
at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:365)
at java.text.SimpleDateFormat.<init>(SimpleDateFormat.java:249)
Here is my code
public static int getDayNumberOfWeek(int day, String monthString, int yyyy) {
//http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html
int dayNumberOfWeek = 1;
final String inputFormat = "MMM/dd/yyyy";
final String outputFormat = "u";
String dayString2Digit = DateTimeHelper.getTwoDigit(day);
String inputTimeStamp = monthString + "/" + dayString2Digit + "/" + String.valueOf(yyyy);
try {
dayNumberOfWeek =Integer.valueOf(TimeStampConverter(inputFormat, inputTimeStamp,
outputFormat));
}
catch (ParseException e) {
e.printStackTrace();
}
return dayNumberOfWeek;
}

I use java 1.7.25
No, you don't - not if you're running on Android. You need to look at the Android documentation, not the Java 7 docs.
If you look at the Android SimpleDateFormat documentation you'll see that u isn't listed there. I don't believe there's a format pattern character for "day of week as a number" in Android.
Were you really looking for that though? If you just want the day of the week as a number (without anything else) you can always use
String text = String.valueOf(calendar.get(Calendar.DAY_OF_WEEK));

If you're using android, then you're not using Java 1.7.25. See the android documentation: there's no support for u in SimpleDateFormat.

I'm guessing your problem is going to be in your TimeStampConverter class where you're passing in that "u" as the outputFormat. "u" is not a valid format character in SimpleDateFormat and you must be constructing a format string that contains it.
If you need to use the "u" as a literal, you'll need to enclose it in single quotes.