We Keep Coding

Conversion Error in TEIID during function call

I have following function defined in a java jar file within the vdb folder:
public static Blob createSampleLogCurve(String indexType, String indexUnit, String curveName, String curveUnit, String curveDataType, Object depthArray, Object valueArray) throws BulkDataException, SQLException { NullValueDef nullDef = new NullValueDef();
byte[] bytes = null;
nullDef.setNullFloat(-98765.0F);
double[] depths = (double[])((double[])depthArray);
int totalsamples = depths.length;
String dataType = DataType.FLOAT.toString();
double increment = depths[1] - depths[0];
for(int i = 0; i < totalsamples - 1; ++i) {
if (increment != depths[i + 1] - depths[i]) {
increment = 0.0D;
break;
}
}
if (curveDataType.equalsIgnoreCase("INT")) {
dataType = DataType.INT.toString();
} else if (curveDataType.equalsIgnoreCase("DOUBLE")) {
dataType = DataType.DOUBLE.toString();
} else {
dataType = DataType.FLOAT.toString();
}
CurveIndexProtoBuf indexProtobuf = new CurveIndexProtoBuf(depths[0], increment, totalsamples, indexType, indexUnit, depths);
List<CurveValueProtoBuf> values = new ArrayList();
values.add(new CurveValueProtoBuf(SampleType.VALUE.toString(), dataType, (String)null, curveUnit, 1, 1, (List)null, curveName, (Index)null, (Index)null, valueArray));
LogCurveProtoBuf logCurveProtobuf = new LogCurveProtoBuf(indexType.equals("DEPTH") ? IndexType.DEPTH.toString() : IndexType.TIME.toString(), indexProtobuf, values);
PipedInputStream in = new PipedInputStream(1024);
PipedOutputStream out = null;
try {
out = new PipedOutputStream(in);
logCurveProtobuf.build().writeDelimitedTo(out);
out.close();
bytes = IOUtils.toByteArray(in);
} catch (Exception var29) {
System.out.println(var29);
} finally {
try {
out.close();
in.close();
} catch (IOException var28) {
var28.printStackTrace();
}
}
return new SerialBlob(bytes)
}
This is how I have declared the UDF in DDL
CREATE VIRTUAL FUNCTION createSampleLogCurve(indexType string, indexUnit string, curveName string, curveUnit string, curveDataType string, depthArray double[], valueArray float[]) returns Blob
OPTIONS(JAVA_CLASS 'com.common.udf.ProtoBufFunctions', JAVA_METHOD 'createSampleLogCurve');
When I call the function from SQL client, I get below error. I also tried object and object[]
java.lang.ClassCastException: org.teiid.core.types.ArrayImpl cannot be cast to [D
Appreciate any help on this
Thanks

The issue is with this line
double[] depths = (double[])((double[])depthArray);
Either try changing the method signature to use double[] depthArray if that does not work, try casting to ArrayImpl like
either one of the above solutions will work!
if (depthArray instanceof ArrayImpl) {
ArrayImpl array = (ArrayImpl) depth array;
Object[] arrayVals = array.getValues();
for (int i = 0; i < arrayVals.length; i++) {
double val = arrayVals[i];
// do
}
}
one of the above solutions should work!

Speed up reading CSV in Java

I have a relatively inefficent CSVReader code, see below. It takes more than 30 seconds to read 30000+ lines. How to speed up this reading process as fast as possible?
public class DataReader {
private String csvFile;
private List<String> sub = new ArrayList<String>();
private List<List> master = new ArrayList<List>();
public void ReadFromCSV(String csvFile) {
String line = "";
String cvsSplitBy = ",";
try (BufferedReader br = new BufferedReader(new FileReader(csvFile))) {
System.out.println("Header " + br.readLine());
while ((line = br.readLine()) != null) {
// use comma as separator
String[] list = line.split(cvsSplitBy);
// System.out.println("the size is " + country[1]);
for (int i = 0; i < list.length; i++) {
sub.add(list[i]);
}
List<String> temp = (List<String>) ((ArrayList<String>) sub).clone();
// master.add(new ArrayList<String>(sub));
master.add(temp);
sub.removeAll(sub);
}
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(master);
}
public List<List> getMaster() {
return master;
}
}
UPDATE: I have found that my code actually can finish the reading work in less than 1 second if run it separately. As this DataReader is a part used by my simulation model to initialize the relevant properties. And the following part is associated with the use of the data imported, WHICH TAKES 40 SECONDS TO FINISH! Anyone could help by looking at the generic part of the codes?
// add route network
Network<Object> net = (Network<Object>)context.getProjection("IntraCity Network");
IndexedIterable<Object> local_hubs = context.getObjects(LocalHub.class);
for (int i = 0; i <= CSV_reader_route.getMaster().size() - 1; i++) {
String source = (String) CSV_reader_route.getMaster().get(i).get(0);
String target = (String) CSV_reader_route.getMaster().get(i).get(3);
double dist = Double.parseDouble((String) CSV_reader_route.getMaster().get(i).get(6));
double time = Double.parseDouble((String) CSV_reader_route.getMaster().get(i).get(7));
Object source_hub = null;
Object target_hub = null;
Query<Object> source_query = new PropertyEquals<Object>(context, "hub_code", source);
for (Object o : source_query.query()) {
if (o instanceof LocalHub) {
source_hub = (LocalHub) o;
}
if (o instanceof GatewayHub) {
source_hub = (GatewayHub) o;
}
}
Query<Object> target_query = new PropertyEquals<Object>(context, "hub_code", target);
for (Object o : target_query.query()) {
if (o instanceof LocalHub) {
target_hub = (LocalHub) o;
}
if (o instanceof GatewayHub) {
target_hub = (GatewayHub) o;
}
}
// System.out.println(target_hub.getClass() + " " + time);
// Route this_route = (Route) net.addEdge(source_hub, target_hub);
// context.add(this_route);
// System.out.println(net.getEdge(source_hub, target_hub));
if (net.getEdge(source, target) == null) {
Route this_route = (Route) net.addEdge(source, target);
context.add(this_route);
// this_route.setDist(dist);
// this_route.setTime(time); }
}
}

In your code you are doing many write operation to just add the list of values from current row in your master list which is not required. You can replace the existing code with simple one as given below.
Existing code:
String[] list = line.split(cvsSplitBy);
// System.out.println("the size is " + country[1]);
for (int i = 0; i < list.length; i++) {
sub.add(list[i]);
}
List<String> temp = (List<String>) ((ArrayList<String>) sub).clone();
// master.add(new ArrayList<String>(sub));
master.add(temp);
sub.removeAll(sub);
Suggested code:
master.add(Arrays.asList(line.split(cvsSplitBy)));

I don't have a CSV that big, but you could try the following:
public static void main(String[] args) throws IOException {
Path csvPath = Paths.get("path/to/file.csv");
List<List<String>> master = Files.lines(csvPath)
.skip(1)
.map(line -> Arrays.asList(line.split(",")))
.collect(Collectors.toList());
}
EDIT: I tried it with a CSV sample with 50k entries and the code runs in less than one second.

With extends to the answer of #Alex R, you can process it in parallel as well like this:
public static void main(String[] args) throws IOException {
Path csvPath = Paths.get("path/to/file.csv");
List<List<String>> master = Files.lines(csvPath)
.skip(1).parallel()
.map(line -> Arrays.asList(line.split(",")))
.collect(Collectors.toList());
}

How to apply for loop on first n vlalues of arraylist

I have an arraylist in which I have ESSID, BSSID, Strenght of access Point on first three indexes, and from Index 4 to 6 I have again ESSID, BSSID, Strength of another AccessPoint. I want to store this list in database like first three values save in one row of table. and next three values save in 2nd row of table.
String[] namesArr = new String[arrayList2.size()]; //conver arraylist to array
for (int j = 0; j < arrayList2.size(); j++){
namesArr[j] = arrayList2.get(j);
int length = namesArr[j].length();
for (int k = 0; k < length; k += 3) {
ssid = namesArr[k];
bssid = namesArr[k + 1];
rssid = namesArr[k + 2];
}
insertValues(this);
}
public void insertValues(View.OnClickListener view){
SendData send = new SendData(this);
send.execute(bssid,ssid,rssid);}
I have made a class to store this data in database that works fine.
public class SendData extends AsyncTask<String, Void, String> {
AlertDialog dialog;
Context context;
public SendData(Context context) {
this.context = context;
}
#Override
protected void onPreExecute() {
dialog = new AlertDialog.Builder(context).create();
dialog.setTitle("Message");
}
#Override
protected void onPostExecute(String s) {
dialog.setMessage(s);
dialog.show();
}
#Override
protected String doInBackground(String... voids) {
String data = "";
String result = "";
String MAC = voids[0];
String Name = voids[1];
String Strength = voids[2];
String con_Str = "http://10.5.48.129/Webapi/accesspoints_data/create.php";
try{
URL url = new URL(con_Str);
HttpURLConnection http = (HttpURLConnection) url.openConnection();
http.setRequestMethod("POST");
http.setDoInput(true);
http.setDoOutput(true);
OutputStream out_Stream = http.getOutputStream();
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(out_Stream, "UTF-8"));
JSONObject obj = new JSONObject();
try {
obj.put("BSSID", MAC);
obj.put("ESSID", Name);
obj.put("RSSID", Strength);
} catch (JSONException e) {
e.printStackTrace();
}
data = obj.toString();
writer.write(data);
writer.flush();
writer.close();
out_Stream.close();
InputStream in_Stream = http.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in_Stream, "ISO-8859-1"));
String line = "";
while ((line = reader.readLine()) != null)
{
result += line;
}
reader.close();
in_Stream.close();
http.disconnect();
return result;
} catch (MalformedURLException e){
result = e.getMessage();
} catch (IOException e){
result = e.getMessage();
}
return result;
}
}
SendData class is perfectly working but problem is with for loop.

I think this is result that you are expecting :
List<String> arrayList2 = new ArrayList<>();
arrayList2.add("1");
arrayList2.add("2");
arrayList2.add("3");
arrayList2.add("4");
arrayList2.add("5");
arrayList2.add("6");
arrayList2.add("6");
arrayList2.add("7");
arrayList2.add("8");
arrayList2.add("9");
arrayList2.add("10");
List<String[]> sarrayList = new ArrayList<>();
String[] arr = new String[3];
int i = 0;
for (int j = 0; j < arrayList2.size(); j++)
{
arr[i] = arrayList2.get(j);
i++;
if((j+1)%3==0)
{
sarrayList.add(arr);
i = 0;
arr = new String[3];
}
}
for(String [] sa:sarrayList)
{
for(String s:sa)
{
System.out.println(s);
}
System.out.println("=========");
}
This might not be the most efficient way of doing it. But it splits the ArrayList in to String arrays of length=3 and stores them in a new ArrayList named sarrayList

I would advise to use a datastructure to hold the record. See the code below this is a small example how you could do it
ArrayList<Record> records;
for (int i = 2; i < inputArrayList.size(); i = i + 3){
string ssid = namesArr.get(i - 2);
string bssid = namesArr.get(i - 1);
string rssid = namesArr.get(i);
records.add(new Record(ssid, bssid, rssid));
}
class Record{
string ssid;
string bssid;
string rssid;
// Constructor...
// Getter and setter to be implemented...
}

ok from what i understand you want to divide the arraylist each 3 elements thats how you do it with streams and it will return an a collection of arraylists each one has 3 elements
final int chunkSize = 3;
final AtomicInteger counter = new AtomicInteger();
//arrayList here us your array list
final Collection<List<String>> result = arrayList.stream()
.collect(Collectors.groupingBy(it -> counter.getAndIncrement() / chunkSize))
.values();
and mentioning supermar10 answer you code make a class to map the strings to it like that
class Record{
string ssid;
string bssid;
string rssid;
Record(String ssid,String bssid,String rssid){
this.ssid=ssid;
this.bssid=bssid;
this.rssid=rssid;
}
}
now you have a class to map to now save the records in a list of Record
create a a list in the main class
static List<Record> lists=new ArrayList<>();
then map the data like that
result.stream().forEach(nowList -> saveRecord(nowList));
and thats the save method
static void saveRecord(List<String> list){
lists.add(new Record(list.get(0),list.get(1),list.get(2)));
}

I have simplified it to one loop and also modified insertValues so that it takes 3 more parameters. This
int size = arrayList2.size();
for (int j = 0; j < size; j += 3) {
if (size - j < 3 ) {
break;
}
String ssid = arrayList2.get(j);
String bssid = arrayList2.get(j + 1);
String rssid = arrayList2.get(j + 2);
insertValues(this, ssid, bssid, rssid);
}
if one the other hand ssid and so on are class variables the inside of the loop can be changed to
ssid = arrayList2.get(j);
bssid = arrayList2.get(j + 1);
rssid = arrayList2.get(j + 2);
insertValues();

How do I properly parse this JSON?

I have a big JSON file that contains Countries, States and Cities. The JSON looks like this:
{
"Countries":[
{
"CountryName":"India",
"States":[
{
"StateName":"Maharashtra",
"Cities":[
"Pune",
"Nagpur",
"Mumbai"
]
},
{
"StateName":"Kerala",
"Cities":[
"Kochi",
"Munnar"
]
}
]
},
{
"CountryName":"Australia",
"States":[
{
"StateName":"Aukland",
"Cities":[
"GlenField",
"Henderson",
"MilFord"
]
},
{
"StateName":"Melbourne",
"Cities":[
"Melbourne",
"South Oakleigh"
]
}
]
}
]
}
I have an activity in my app that requires the user to select a country and upon selecting a country I want to get all the states of that country as an array list. And when a specific state is selected, I want to get all the cities of that state as an array list.
I am able to get a list of all Countries using this method:
public void loadCountries(String parent, String child, ArrayList<String> listValue)
{
try {
JSONObject obj = new JSONObject(loadJSONFromAsset());
JSONArray m_jArry = obj.getJSONArray(parent);;
ArrayList<HashMap<String, String>> formList = new ArrayList<HashMap<String, String>>();
HashMap<String, String> m_li;
// listValue = new ArrayList<>();
for (int i = 0; i < m_jArry.length(); i++) {
JSONObject jo_inside = m_jArry.getJSONObject(i);
Log.d("Details-->", jo_inside.getString(child));
listValue.add(jo_inside.getString(child));
}
} catch (JSONException e) {
e.printStackTrace();
}
}
public String loadJSONFromAsset() {
String json = null;
try {
InputStream is = getAssets().open("Contries.json");
int size = is.available();
byte[] buffer = new byte[size];
is.read(buffer);
is.close();
json = new String(buffer, "UTF-8");
} catch (IOException ex) {
ex.printStackTrace();
return null;
}
return json;
}
Then, in the onCreate method, using this line gives me all the countries in an array list
loadCountries("Countries", "CountryName", countries);
Since this is my first time working with a JSON file in AndroidStudio/Java, I have no idea what to do to get the states and the cities. Any help is greatly appreciated.

You could use GSON library. Just create corresponding java object which will reflect json structure and do like here:
MyClass data = new Gson().fromJson(json, MyClass.class);

Respect for trying to do this yourself, but we live in the age of the API. Check out Google's GSON library, it parses JSON to java objects, and java objects to JSON. It's one of the most useful APIs out there. https://github.com/google/gson

Try this:
Map<String, Map<String, Map<Integer, String>>> data = new HashMap<>();
try {
JSONObject object = new JSONObject(loadJSONFromAsset());
JSONArray countries = object.getJSONArray("Countries");
for (int i = 0; i < countries.length(); i++) {
JSONObject state = countries.getJSONObject(i);
JSONArray statesList = state.getJSONArray("States");
String countryName = countries.getJSONObject(i).getString("CountryName");
for (int j = 0; j < statesList.length(); j++) {
JSONObject city = statesList.getJSONObject(j);
JSONArray citiesList = city.getJSONArray("Cities");
String stateName = city.getString("StateName");
Map<String, Map<Integer, String>> stateHashMap = new HashMap<>();
for (int k = 0; k < citiesList.length(); k++) {
cityList.add(citiesList.getString(k));
String cityName = citiesList.getString(k);
Map<Integer, String> cityHashmap = new HashMap<>();
cityHashmap.put(k, cityName);
stateHashMap.put(stateName, cityHashmap);
data.put(countryName, stateHashMap);
}
}
}
} catch (JSONException e) {
e.printStackTrace();
}

CSV is not getting read properly in java using csvreader

This is my code. I am not able to write it back to a csv. I have figured out that the problem is with the header matching in the if loop. The values are being read correctly nut the if loop is not getting satisfied when the headers are getting matched. Any help is much appreciated. Thanks.
public class FactsChecker {
public static void main(String[] args) {
run();
}
static void run()
{
Map <String,String> csvMap = new HashMap<String,String>();
String csvPath = "resource/Sample.csv";
String outputPath = "resource/Output.csv";
String[] headers;
try{
CsvReader csvreader = new CsvReader("resource/Sample.csv");
csvreader.readHeaders();
headers = csvreader.getHeaders();
String[] values;
String key = null;
String value = null;
while (csvreader.readRecord()){`
values = csvreader.getValues();
System.out.println(headers[0]);
for (int i = 0; i < headers.length; i++) {
if (headers[i].equals("FCT_NAME")) {
key = values[i];
}
if (headers[i].equals("FCT_TYPE")) {
value = values[i];
}
}
csvMap.put(key,value);
}
for (Map.Entry<String, String> entry : csvMap.entrySet())
{
System.out.println(entry.getKey() + "/" + entry.getValue());
}
Map<String,Entry> entryMap= new HashMap<String, Entry>();
for (String factKey:csvMap.keySet()){
Entry entry = new Entry();
entry.setFactName(factKey);
String factType = csvMap.get(factKey);
System.out.println(factKey);
entry.setFactType(factType);
entryMap.put(factKey, entry);
}
CsvWriter writer = new CsvWriter(outputPath);
for(String key1 : entryMap.keySet()){
Entry entry = entryMap.get(key1);
writer.writeRecord(new String[]{entry.getFactName(),entry.getFactType()});
}
writer.close();
}
catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}

try using contains instead of equals in case there is trailing or leading white space

I also faced the same issue .Try the below. Just remove this appended character from your header when you read it from csv and then compare.
if(headers[i].startswith("\uFEFF"){
headers[i].replace("\uFEFF","");
}