Hive UDTF returning ArrayList Column - java

I am new to Hive UDTF. I have a requirement where I have to pass string values as Paratmeter in UDTF and the returning Column should be a ArrayList.
I have written the following Code:
public StructObjectInspector initialize(ObjectInspector[] arg0)
throws UDFArgumentException {
ArrayList<String> fieldNames = new ArrayList<String>();
ArrayList<ObjectInspector> fieldOIs = new ArrayList<ObjectInspector>();
fieldNames.add("col1");
stringOI = (PrimitiveObjectInspector) arg0[0];
listOi=(ListObjectInspector) arg0[0];
fieldOIs.add(listOi.getListElementObjectInspector());
return ObjectInspectorFactory.getStandardStructObjectInspector(fieldNames, fieldOIs);
}
#Override
public void process(Object[] record) throws HiveException {
// TODO Auto-generated method stub
String document = (String) stringOI.getPrimitiveJavaObject(record[0]);
if (document == null) {
return;
}
firstColumn=(String) stringOI.getPrimitiveJavaObject(record[0]);
secondColumn=(String) stringOI.getPrimitiveJavaObject(record[1]);
if(outputMapper.containsKey(firstColumn))
{
ArrayList<String> tempList=new ArrayList<String>();
tempList=outputMapper.get(firstColumn);
tempList.add(secondColumn);
outputMapper.put(firstColumn,tempList);
}
else
{
childVendorList=new ArrayList<String>();
childVendorList.add(secondColumn);
outputMapper.put(firstColumn,childVendorList);
}
forward(outputMapper.get(firstColumn));
}
}
And I am getting the following Exception:
java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyStringObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector
Can Anyone Help???

listOi=(ListObjectInspector) arg0[0];
fieldOIs.add(listOi.getListElementObjectInspector());
return ObjectInspectorFactory.getStandardStructObjectInspector(fieldNames, fieldOIs);
This arg0[0] is a primitive object inspector. With listOi.getListElementObjectInspector(), just get a similar PrimitiveObjectInspector(like String,Integer is not a List). It should
fieldOIs.add(ObjectInspectorFactory.getStandardListObjectInspector(stringOI ))
This will specific the output column with List of type of stringOI.

Related

How to create Async Task to build a RSSReader

Before getting down vote. Yes I read the forums before asking this question. RSSReader Async Task
Read that one above but I still don't get it.
The question:
I wrote een RSSReader in Java. This perfectly works in the console prints what I want etc. But in Android it doesn't work because it's not using een Async Task. Now I understood from the Google Documentation that there are three types to be entered AsyncTask something like that. I have no idea how I can implement this in my code. Do I need to make a seperate class extends it with AsyncTask and create and instance of my Reader and in it's doInBackground method call my reader or how do I need to do this.
This is the code of my RSSReader:
public class RSSReader {
//Lists to store headlines, descriptions & images
String url = "http://www.nu.nl/rss/Algemeen";
List<String> titleList;
List<String> descriptionList;
List<String> imageList;
public RSSReader(){
try {
titleList = readRSS(url, "<title>", "</title>");
descriptionList = listFilter(readRSS(url, "<description>", "</description>"), "&nbsp;", "");
imageList = readRSS(url, "<enclosure url \"", "\" length=\"0\" type=\"image/jpeg\"</enclosure>");
}
catch (IOException e){
}
}
public List<String> readRSS(String feedUrl, String openTag, String closeTag) throws IOException, MalformedURLException {
URL url = new URL(feedUrl);
BufferedReader reader= new BufferedReader(new InputStreamReader(url.openStream()));
String currentLine;
List<String> tempList = new ArrayList<String>();
while((currentLine = reader.readLine()) != null){
Integer tagEndIndex = 0;
Integer tagStartIndex = 0;
while (tagStartIndex >= 0){
tagStartIndex = currentLine.indexOf(openTag, tagEndIndex);
if(tagStartIndex >= 0){
tagEndIndex = currentLine.indexOf(closeTag, tagStartIndex);
tempList.add(currentLine.substring(tagStartIndex + openTag.length(), tagEndIndex) + "\n");
}
}
}
tempList.remove(0);
return tempList;
}
public List<String> getDesciptionList(){
return descriptionList;
}
public List<String> getTitleList(){
return titleList;
}
public List<String> getImageList(){
return imageList;
}
public List<String> listFilter(List<String> tempList, String require, String
replace){
//Creates new List
List<String> newList = new ArrayList<>();
//Loops through old list and checks for the 'require' variable
for(int i = 0; i < tempList.size(); i++){
if(tempList.get(i).contains(require)){
newList.add(tempList.get(i).replace(require, replace));
}
else{
newList.add(tempList.get(i));
}
}
return newList;
}
}
In RSSReader#readRSS,you do not check tempList.size()
and do not forget add
<uses-permission android:name="android.permission.INTERNET"/>
to your AndroidManifest.xml
for example
public class MainActivity extends Activity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
new RssReaderAsyncTask(new RSSCallBack() {
#Override
public void success(RSSReader rssReader) {
// TODO That Should run on UI Thread if you update UI
// for example
final RSSReader reader = rssReader;
// you can use runOnUiThread or Handler update UI
runOnUiThread(new Runnable() {
#Override
public void run() {
// TODO Toast
Toast.makeText(MainActivity.this, reader.getTitleList().toString(), Toast.LENGTH_SHORT).show();
}
});
}
#Override
public void failed() {
// TODO That Should run on UI Thread if you update UI
Log.e("RSS", "failed");
}
}).execute("");
}
private class RssReaderAsyncTask extends AsyncTask<String, Integer, Integer> {
private RSSCallBack rssCallBack;
public RssReaderAsyncTask(RSSCallBack rssCallBack) {
this.rssCallBack = rssCallBack;
}
#Override
protected Integer doInBackground(String... params) {
// TODO
try {
RSSReader reader = new RSSReader();
rssCallBack.success(reader);
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
rssCallBack.failed();
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
rssCallBack.failed();
e.printStackTrace();
}
return null;
}
}
private interface RSSCallBack {
void success(RSSReader rssReader);
void failed();
}
public class RSSReader {
// Lists to store headlines, descriptions & images
String url = "http://www.nu.nl/rss/Algemeen";
List<String> titleList;
List<String> descriptionList;
List<String> imageList;
public RSSReader() throws MalformedURLException, IOException {
titleList = readRSS(url, "<title>", "</title>");
descriptionList = listFilter(readRSS(url, "<description>", "</description>"), "&nbsp;", "");
imageList = readRSS(url, "<enclosure url \"", "\" length=\"0\" type=\"image/jpeg\"</enclosure>");
}
public List<String> readRSS(String feedUrl, String openTag, String closeTag)
throws IOException, MalformedURLException {
URL url = new URL(feedUrl);
BufferedReader reader = new BufferedReader(new InputStreamReader(url.openStream()));
String currentLine;
List<String> tempList = new ArrayList<String>();
while ((currentLine = reader.readLine()) != null) {
Integer tagEndIndex = 0;
Integer tagStartIndex = 0;
while (tagStartIndex >= 0) {
tagStartIndex = currentLine.indexOf(openTag, tagEndIndex);
if (tagStartIndex >= 0) {
tagEndIndex = currentLine.indexOf(closeTag, tagStartIndex);
tempList.add(currentLine.substring(tagStartIndex + openTag.length(), tagEndIndex) + "\n");
}
}
}
if (tempList.size() > 0) {
//TODO you do not check it
tempList.remove(0);
}
return tempList;
}
public List<String> getDesciptionList() {
return descriptionList;
}
public List<String> getTitleList() {
return titleList;
}
public List<String> getImageList() {
return imageList;
}
public List<String> listFilter(List<String> tempList, String require, String replace) {
// Creates new List
List<String> newList = new ArrayList<String>();
// Loops through old list and checks for the 'require' variable
for (int i = 0; i < tempList.size(); i++) {
if (tempList.get(i).contains(require)) {
newList.add(tempList.get(i).replace(require, replace));
} else {
newList.add(tempList.get(i));
}
}
return newList;
}
}
}
You are right, you need Asynctask. But it is too much to explain here, it has already been explained very thoroughly here, so you might wanna take a look:
https://stackoverflow.com/a/9671602/3673616
What you need to make sure is to run your network calls in doInBackground, you can manipulate the UI in onPreExcute and after finish in onpostExecute. For more details please visit the link.
Well i assume that you already know the code so in the doInBackground method should be the long running code, like getting information from internet/server etc. You can then return a string with success or error that will be catched from onPostExecute method, where you can just do what ever you like with the result.
So i would say no need for new class just extend async task in this, implement the 2 methods i mentioned and in the methods call the right function that you already have with just a litttle change for returning result.

Passing List<classname> to ArrayAdapter

The problem is this adapter is giving the error although i have pass the Object array to it.(Read the methods belows then you will find what i want to know from you guys)
This method declares a List of private class objects. Then return that list of object to onPostExecute method.
private class DownloadXmlTask extends AsyncTask<String, Void, String> {
#Override
protected String doInBackground(String... urls) {
try {
return loadXmlFromNetwork(urls[0]);
} catch (IOException e) {
return "I/O exception ae hy";
} catch (XmlPullParserException e) {
return "XML pull parser ke exception ae hy";
}
}
#Override
protected void onPostExecute(List<StackOverflowXmlParser.Entry> result) {
//Log.d(TAG,result.toString());
ArrayAdapter<String> adapter;
adapter = new ArrayAdapter<String>(this,android.R.layout.simple_list_item_1,result);
setListAdapter(adapter);
}
private Object loadXmlFromNetwork(String urlString) throws XmlPullParserException, IOException {
InputStream stream = null;
// Instantiate the parser
StackOverflowXmlParser stackOverflowXmlParser = new StackOverflowXmlParser();
List<StackOverflowXmlParser.Entry> entries = null;
String title = null;
String url = null;
String summary = null;
try {
stream = downloadUrl(urlString);
entries = stackOverflowXmlParser.parse(stream);
} finally {
if (stream != null) {
stream.close();
}
}
for (StackOverflowXmlParser.Entry entry : entries)
{
Log.d(TAG, entry.link + " /" + entry.title);
}
return entries;
}
I think it should be onPostExecute(List<StackOverflowXmlParser.Entry> result)
And you AsyncTask should be
extends AsyncTask<smth, smth, List<StackOverflowXmlParser.Entry> >
ArrayAdapter<String> requires that you provide it a String[] or a List<String>. You are trying to pass in Object[], which is neither String[] nor List<String>. And, it would appear that you are really trying to populate the ListView with a list of StackOverflowXmlParser.Entry objects, which are not String objects.
My guess is that the right answer is for you to create an ArrayAdapter<StackOverflowXmlParser.Entry> instead of an ArrayAdapter<String>.
Regardless, you need to ensure that the data type in your declaration (String in ArrayAdapter<String>) matches the data type in your constructor parameter that supplies the data to be adapted.

How to load ArrayList data into ListView using Android?

I am trying to create JSON data store into ArrayList and load on ListView. Now I have successfully stored my JSON data into ArrayList. But the problem is I have maintaining multiple column listview. I need to List out my first array on first column.
Below I have tried something, multiple columns with array. But exactly I dont know how to do that. Please help me, I am new developer for Android.
// I need to add my array into first column
private ArrayList<String> myarray = new ArrayList<String>();
//JSON string data's I have loaded
myarray.add(jsondata);
//LISTVIEW WATCHLIST
ListView listView=(ListView)findViewById(R.id.listView1);
list=new ArrayList<HashMap<String,String>>();
HashMap<String,String> temp=new HashMap<String, String>();
temp.put(FIRST_COLUMN, "Minchu");
temp.put(SECOND_COLUMN, "USA");
temp.put(THIRD_COLUMN, "City");
temp.put(FOURTH_COLUMN, "Ranks");
list.add(temp);
.
.
.
.
ListViewAdapters adapter=new ListViewAdapters(this,list);
listView.setAdapter(adapter);
NOTE : Above HashMap to putted manual data. I need to load first array first columns, second array second columns.
You need to create model for your json object see below code.
import java.io.Serializable;
public class PersonDetailsItem implements Serializable {
public int id;
private String intEducationEN, intVillageID;
public PersonDetailsItem(int id, String name, String phoneNo, String email) {
// TODO Auto-generated constructor stub
this.id = id;
this.strNameEN = name;
this.strEmailid = email;
}
public String getIntVillageID() {
return intVillageID;
}
public void setIntVillageID(String intVillageID) {
this.intVillageID = intVillageID;
}
public String getIntEducationEN() {
return intEducationEN;
}
public void setIntEducationEN(String intEducationEN) {
this.intEducationEN = intEducationEN;
}
public PersonDetailsItem() {
// TODO Auto-generated constructor stub
}
Then set the value for the model form parsing json received string.
private void parseJson(String rs) {
private ArrayList<PersonDetailsItem> listData = new ArrayList<PersonDetailsItem>();;
// TODO Auto-generated method stub
listData = new ArrayList<PersonDetailsItem>();
spinerPersonData = new ArrayList<SpinerItem>();
try {
JSONObject obj = new JSONObject(rs);
JSONArray jArray = obj.getJSONArray("Table");
for (int i = 0; i < jArray.length(); i++) {
JSONObject c = jArray.optJSONObject(i);
String intEducationEN = c.getString("intEducationEN");
String intVillageID = c.getString("intVillageID");
PersonDetailsItem personItem = new PersonDetailsItem();
personItem.setIntEducationEN(intEducationEN);
personItem.setIntVillageID(intVillageID);
listData.add(personItem);
}
} catch (JSONException e) { // TODO Auto-generated catch block
e.printStackTrace();
Log.v("perosnJson Error", e.toString());
}
}
set to listadapter
CustomAdapter adapter = new CustomAdapter(MainActivity.this, R.id.listView1, listData);
lv.setAdapter(adapter);
adapter.notifyDataSetChanged();
enter code here
You can check link for more detail https://github.com/yagneshshinde101/mysamaj/blob/master/MySamaj/src/com/example/mysamajmain/MainActivity.java

In which container can I collect Strings to show any of them

I have a LinkedHashMap which fills with data from db with loop "for" string by string and when I try to show the first or the last String, the method can show me only the last String in log. But in application listViewContent is filled fully. So I don't understand why I can't see any string that I want. I need to collect all strings I get from db and compare them in future.
How can I collect all strings and what method should I call to show the string I want to see?Unfortunately I can only retrieve one (and the last instead of the first) string.
Here is my example code :
protected void onCreate(Bundle saveInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
FirstMethod();
}
public FirstMethod() {
SecondMethod newMethod = .. // getting data from the second method
}
public SecondMethod() {
public void onResponseReceived(String result) {
try {
...
if (posts != null) {
for (WallPostItem post : posts) { // this loop
//create new map for a post
Map<String, Object> map = new LinkedHashMap<String, Object>();
map.put(ATTRIBUTE_NAME_TEXT, post.text);
PictureItem postPicture = new PictureItem();
map.put(ATTRIBUTE_NAME_IMAGE, postPicture);
map.put(ATTRIBUTE_NAME_DATE, post.date);
sAdapter.notifyDataSetChanged();
};
};
...
List<Map.Entry<String, Object>> list = new ArrayList<Map.Entry<String, Object>>(GlobalMap.entrySet());
Map.Entry<String, Object> firstInsertedEntry = list.get(0);
Log.w("FirstEntryOfMap",""+firstInsertedEntry); // this log shows me the last string instead of the first
}
if (isRefresh) {
isRefresh = false;
lvSimple.setSelectionAfterHeaderView();
}
} catch (Exception e) {
Log.d("exceptions", "problem in get wall post task after post execute: " + e.toString());
}
}
You aren't putting your values into a List, you are putting them into a Map (that preserves key order). I would suggest you create a POJO class,
class MyAttribute {
final String postName;
final PictureItem postPicture;
final Date postDate;
public MyAttribute(String postName, PictureItem postPicture, Date postDate) {
this.postName = postName;
this.postPicture = postPicture;
this.postDate = postDate;
}
public String getPostName() {
return postName;
}
public Date getPostDate() {
return postDate;
}
public PictureItem getPostPicture() {
return postPicture;
}
}
Then you could create a
List<MyAttribute> myAttributes = new ArrayList<>();

Java Boolean into Object[][]

I got following code and there is this error (I tried to keep the code as short as possible, ignore the getColumnCount etc. functions just the constructor):
The following code is used to make a JTable in Swing by an SQLite statement, I need the booleans for checkboxes (Yes I know I have to edit/add an function, but I wanted to keep the code as small as possible).
Code:
package view;
import java.sql.ResultSet;
import java.sql.SQLException;
import javax.swing.table.AbstractTableModel;
import controller.Database;
class Test extends AbstractTableModel {
Database db = new Database();
ResultSet rs;
private String[] columnNames = {"Vorname", "Nachname", "E-Mail", "Anrede", "Jahrgang", "Ablösung", "Scheibe", "Waffe", "Gruppe", "Verpflegung", "Aktiv"};
Object[][] data;
public Test(){
int result = 0;
try {
rs = db.stat.executeQuery("select count(*) as schuetzencount from schuetze;");
result = Integer.parseInt(rs.getString("schuetzencount"));
data = new String[result][11];
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
rs = db.stat.executeQuery("select * from schuetze as s join waffe as w on w.Waffe_ID = s.Waffe_ID join gruppe as g on g.Gruppe_ID = s.Gruppe_ID join anrede as a on a.Anrede_ID = s.Anrede_ID join verpflegung as v on v.Verpflegung_ID = s.Verpflegung_ID;");
int counter = 0;
while(rs.next()){
data[counter][1] = rs.getString("Schuetze_Nachname");
data[counter][0] = rs.getString("Schuetze_Vorname");
data[counter][4] = rs.getString("Schuetze_Jahrgang");
data[counter][2] = rs.getString("Schuetze_Email");
data[counter][5] = rs.getString("Schuetze_Abloesung");
data[counter][6] = rs.getString("Schuetze_Scheibe");
data[counter][7] = rs.getString("Waffe_Name");
data[counter][8] = rs.getString("Gruppe_Name");
data[counter][3] = rs.getString("Anrede_Name");
data[counter][9] = rs.getString("Verpflegung_Name");
data[counter][10] = true;
counter++;
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
#Override
public int getColumnCount() {
// TODO Auto-generated method stub
return 0;
}
#Override
public int getRowCount() {
// TODO Auto-generated method stub
return 0;
}
#Override
public Object getValueAt(int arg0, int arg1) {
// TODO Auto-generated method stub
return null;
}
public static void main(String[] args) {
Test t = new Test();
}
}
Error:
Exception in thread "main" java.lang.ArrayStoreException: java.lang.Boolean
at view.Test.<init>(Test.java:43)
at view.Test.main(Test.java:72)
If I'll do
Object[][] data = {{"Test", "Test","Test","Test","Test","Test","Test","Test","Test","Test",true}}
it works, but that's not what I need. Then I tried to do a Object[] and fill the Booleans in and then add the Object[] into the data[][] but this also didn't work.
I hope someone can help me, thanks.
Greetz.
You have Array of Strings and try to put Boolean there in line
data[counter][10] = true;
That is not allowed.
When You do
Object[][] data = {{"Test", "Test","Test","Test","Test","Test","Test","Test","Test","Test",true}}
Java creates for you array of Objects
Like:
Object[][] o = new Object[1][6];
o[0][2] = true; // it works
Your array expects a String as dataType, on the other hand are you trying to put a boolean into it.
data[counter][10] = true;
a simple solution to this is using a string "true"instead of the primitive booleantype (or parse it to string)
data[counter][10] = "true";
You cannot insert a boolean value to an array of String. That is the problem :
data = new String[result][11];
data[counter][10] = true;
At least insert it as a String and parse it when neccessary.
You have defined data as matrix of Objects, but instantiated it as matrix of String.
That's why, at runtime, you have this type mismatch exception.
Either instantiate it like:
Object[][] o = new Object[1][6];
or
replace boolean value with string:
data[counter][10] = "true";
The problem is here:
data = new String[result][11];
You declare the array of String and you do this
data[counter][10] = true;
You have two options:
declare data as
new Object[result][11];
put string instead of boolean
data[counter][10] = Boolean.TRUE.toString();

Categories