I need to load the data to the elasticsearch index. I am using BULK API of elasticsearch to load the JSONs to index.
private String FOLDER_PATH = "src/main/resources/allJsons";
private String index = "test1";
private static final String TYPE = "test_type";
#Autowired
private RestHighLevelClient restHighLevelClient;
public String loadBulkData() throws IOException {
BulkRequest bulkRequest = new BulkRequest();
AtomicInteger counter = new AtomicInteger();
try (Stream<Path> filePathStream = Files.walk(Paths.get(FOLDER_PATH))) {
filePathStream.forEach(filePath -> {
if (Files.isRegularFile(filePath)) {
counter.getAndIncrement();
try {
String content = Files.readString(filePath);
JSONObject jsonObject1 = new JSONObject(content);
HashMap yourHashMap1 = new Gson().fromJson(jsonObject1.toString(), HashMap.class);
IndexRequest indexRequest = new IndexRequest(index, TYPE).source(yourHashMap1);
bulkRequest.add(indexRequest);
} catch (IOException e) {
e.printStackTrace();
}
}
});
}
try {
restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
e.printStackTrace();
}
return "Bulk data loaded to index " + index + "";
}
}
I have multiple JSONs based on the following format
[
{
"Nutrient" : "Calories",
"Amount" : " 289.00",
"Unit" : " kcal"
}, {
"Nutrient" : "Fat",
"Amount" : " 17.35",
"Unit" : " g"
}
]
While running the code it gives me error ,
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.json.JSONException: A JSONObject text must begin with '{' at 1 [character 2 line 1]
I think the data is in JSONArray and for the code, we need JSONObject. Anyone could please guide to how to do this
You can do bulk insertion by passing hashmap of your json objects to Elasticsearch Bulk API.
You can create Hashmap by parsing your JSON file through JSONParser.
Here is the code for the same :
Code :
Integer id= 1;
//You need to call this method for inserting bulk documents which
// internally calls `createBulkRequest` and `parseObjectList` methods.
//This method uses JSONParser to parse your file and convert into JSONArray.
public String insertBulkDocuments() throws Exception {
Object obj = new JSONParser().parse(new FileReader(<path-of-file>));
JSONArray objList= (JSONArray) obj;
BulkRequest request = createBulkRequest(objList);
BulkResponse bulkresp=restHighLevelClient.bulk(request, RequestOptions.DEFAULT);
return bulkresp.status().toString();
}
// Each JSONArray element that was obtained through first method
//is parsed individually through Gson and converted into you defined Object.
//This object is then converted to Map and passed to IndexRequest object.
private BulkRequest createBulkRequest(JSONArray objList) {
BulkRequest request = new BulkRequest();
objList.forEach( obj -> parseObjectList((JSONObject) obj, request,id++));
return request;
}
private void parseObjectList(JSONObject obj, BulkRequest request, int id) {
Gson gson = new GsonBuilder().create();
NutrientDocument doc = gson.fromJson(obj.toJSONString(), NutrientDocument .class);
Map<String, Object> documentMapper = objectMapper.convertValue(doc, Map.class);
IndexRequest indexRequest = new IndexRequest(<your-index-name>).id(Integer.toString(id)).source(documentMapper);
request.add(indexRequest);
}
You need to create Custom object which has same feilds as your json . I have created NutrientDocument for testing which has same fields as your JSON and this I am using in parseObjectList method.
public class NutrientDocument {
private String Nutrient;
private Float Amount;
private String Unit;
public String getNutrient() {
return Nutrient;
}
public void setNutrient(String nutrient) {
Nutrient = nutrient;
}
public Float getAmount() {
return Amount;
}
public void setAmount(Float amount) {
Amount = amount;
}
public String getUnit() {
return Unit;
}
public void setUnit(String unit) {
Unit = unit;
}
}
NOTE :
For each document elasticserach generates unique id .
For creating our own id value instead of Elasticsearch autogenerated value, we are using id variable. But, if you want to go with Elasticsearch autogenerated number , you can create IndexRequest object as below in parseObjectList method and remove id variable wherever we are passing.
IndexRequest indexRequest = new IndexRequest().source(documentMapper);
Related
I'm using this method that returns a Set<String> but in fact what I got is a Json string like this
[
{
"id":"Id1"
},
{
"id":"Id2",
"title":"anyTitle"
}
]
My goal is to get the value of key "id". I've also made a java bean to map the data:
public class Data {
private String id;
private String title;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getTitle() {
return title;
}
public void setTitle(String title) {
this.title = title;
}
}
I tryied to parse using gson but all I can get is an error: Cannot cast 'java.util.LinkedHashMap$LinkedKeyIterator' to 'com.google.gson.stream.JsonReader'
So, obviously I'm doing something wrong:
Set<String> availableData = getData(); //this method returns a json string
Iterator<String> itr = availableData.iterator();
while (itr.hasNext()) {
Gson gson = new Gson();
JsonParser parser = new JsonParser();
JsonObject object = (JsonObject) parser.parse(itr.next());
Data data = gson.fromJson(object, Data.class);
}
update: The actual error is: Type mismatch Can't assign com.google.common.collect.Maps$TransformedEntriesMap to java.lang.String
In that line you pass an iterator:
JsonObject object = (JsonObject) parser.parse((JsonReader) itr);
But you should pass a next element:
JsonObject object = (JsonObject) parser.parse(itr.next());
In addition you got an extra comma in you JSON.
You can replace the whole block with that line:
Data data = gson.fromJson(itr.next(),Data.class)
Use Jackson mapper. You can directly convert it into an object and retrieve through getters.
ObjectMapper objectMapper = new ObjectMapper();
String carJson =
"{ \"brand\" : \"Mercedes\", \"doors\" : 5 }";
try {
Car car = objectMapper.readValue(carJson, Car.class);
System.out.println("car brand = " + car.getBrand());
System.out.println("car doors = " + car.getDoors());
} catch (IOException e) {
e.printStackTrace();
}
So, following this related issue: https://github.com/seleniumhq/selenium-google-code-issue-archive/issues/5154, finally I map this using JSONArray and streams from java8
Set<String> availableData = getData();
JSONArray dataArray = new JSONArray(availableData);
List<Object> dataList = dataArray.toList();
Object o = dataList.stream()
.filter(c -> ((Map) c).get("id").toString().contains("Id1"))
.findFirst().orElse(null);
return ((Map)o).get("id").toString();
Maybe you want to known how to use Gson to unserialized json to java object.
Here are two ways I can give you.
public void parse() {
String jsonString = "[\n" +
" {\n" +
" \"id\":\"Id1\"\n" +
" },\n" +
" {\n" +
" \"id\":\"Id2\",\n" +
" \"title\":\"anyTitle\"\n" +
" }\n" +
"]";
Gson gson = new Gson();
// Use Gson Type
Type setType = new TypeToken<HashSet<Data>>(){}.getType();
Set<Data> dataSet = gson.fromJson(jsonString, setType);
// Print [Data{id='Id2', title='anyTitle'}, Data{id='Id1', title='null'}]
System.out.println(dataSet);
// Use Java Array
Data[] dataArray = gson.fromJson(jsonString, Data[].class);
// Print [Data{id='Id1', title='null'}, Data{id='Id2', title='anyTitle'}]
System.out.println(Arrays.toString(dataArray));
}
I am using below code to automate REST API.
Please help me to understand how can I put whole json data for sample data mentioned below as the input has arrays whereas till now I used flat jsons without arrays
Method Dummy()
{
RestAssured.baseURI ="http://mydummyURL";
RequestSpecification request = RestAssured.given();
JSONObject requestParams = new JSONObject();
requestParams.put("id", "THAILAND"); //Issue is with this code
request.header("Content-Type", "application/json");
request.body(requestParams.toJSONString());
Response response = request.post("/EndPoint");
}
where the json body looks like this
{
"tag1": "value1",
"tag2": "value2",
"tag3": {
"tag31": "value31",
"tag32": "value32"
},
"tag4": [{
"domainName": "ABC",
"domainId": "123ABC123",
"domainGUID": "TestMyDomain"
},
{
"domainName": "XYZ",
"domainId": "123XYZ123",
"domainGUID": "TestMyDomain"
}
]
}
ArrayList<JSONObject> array= new ArrayList<JSONObject>();
JSONObject json= new JSONObject();
try {
json.put("key", "value");// your json
} catch (JSONException e) {
e.printStackTrace();
}
array.add(json);
String printjsonarray= array.toString();// pass this into the request
ObjectMapper mapper = new ObjectMapper();
//Create a Java Class for the variables inside array.
JsonArrayData tag4paramVal1 = new JsonArrayData("ABC","123ABC123","TestMyDomain");
JsonArrayData tag4paramVal2 = new JsonArrayData("XYZ","123XYZ123","TestMyDomain");
Object[] tag4ValArray = {tag4paramVal1,tag4paramVal2};
String reqJson = null;
List<String> tag4Data = new ArrayList<String>();
for(Object obj:tag4ValArray){
reqJson = mapper.writeValueAsString(obj);
System.out.println(reqJson);
tag4Data.add(reqJson);
}
System.out.println(tag4Data);
HashMap<String,List<String>> finalReq = new HashMap<String,List<String>>();
finalReq.put("\"tag4\":",tag4Data);
String finalreqString = finalReq.toString();
System.out.println(finalreqString);
finalreqString = finalreqString.replace('=', ' ');
System.out.println(finalreqString);
//Use the above String as a parameter to POST request. You will get your desired JSON array .
//JsonArrayData class code
public class JsonArrayData {
String domainName;
String domainId;
String domainGUID;
public JsonArrayData(String domainName,String domainId,String domainGUID){
this.domainName = domainName;
this.domainId = domainId;
this.domainGUID = domainGUID;
}
public String getDomainName() {
return domainName;
}
public void setDomainName(String domainName) {
this.domainName = domainName;
}
public String getDomainId() {
return domainId;
}
public void setDomainId(String domainId) {
this.domainId = domainId;
}
public String getDomainGUID() {
return domainGUID;
}
public void setDomainGUID(String domainGUID) {
this.domainGUID = domainGUID;
}
}
I have a json file, for example:
{
"A":"-0.4",
"B":"-0.2",
"C":"-0.2",
"D":"X",
"E":"0.2",
"F":"0.2",
"J":"0.3"
}
I want return each element of a list json when I call it via my function.
I did a function to do this:
public JSONObject my_function() {
JSONParser parser = new JSONParser();
List<JSONObject> records = new ArrayList<JSONObject>();
try (FileReader reader = new FileReader("File.json")) {
//Read JSON file
Object obj = parser.parse(reader);
JSONObject docs = (JSONObject) obj;
LOGGER.info("read elements" + docs); // it display all a list of a json file like this: {"A":"-0.4","B":"-0.2","C":"-0.2","D":"X","E":"0.2","F":"0.2","J":"0.3"}
for (int i = 0; i < docs.size(); i++) {
records.add((JSONObject) docs.get(i));
System.out.println((records)); // it return null
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (ParseException e) {
e.printStackTrace();
}
LOGGER.info("The first element of a list is:" +records.get(0)); // return null
return records.get(0);
How can I change my function to return the value of each element in a json file.
For example, when I call my_function:
my_function.get("A") should display -0.4
Thank you
First you need a Class for mapping
public class Json {
private String a;
private String b;
private String c;
private String d;
private String e;
private String f;
private String j;
//getters and setters
}
Then in your working class
ObjectMapper mapper = new ObjectMapper();
//JSON from file to Object
Json jsn = mapper.readValue(new File("File.json"), Json.class);
then you can use that object in a usual way...
here is the dependency I used
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.6.3</version>
</dependency>
Reference
In Java you can use only class`s methods, as I know.
If you want to get your second element by its first, you should in your class create 2 methods like
class Main {
Map<String, String> records = new HashMap<>();
public JSONObject my_function() {
// your realization where you should insert your pairs into Map.
}
public String get(String firstElement){
return map.getValue(firstElement);
}
}
class someOtherClass {
Main main = new Main();
main .my_function();
main .get("A");
}
I am trying to write a test case for the service using Junit Mockito.
Controller class:
public class ReporteesService {
ReporteeList reportee = new ReporteeList();
GetEmpId EmpId = new GetEmpId();
public ReporteesService(ReporteeList reportee) {
this.reportee = reportee;
}
public ReporteesService(EmpJiraList NumberOfJiras) {
this.NumberOfJiras = NumberOfJiras;
}
public ReporteesService(GetEmpId EmpId) {
this.EmpId = EmpId;
}
#GET
#Path("/ReporteeList/{empid}")
#Produces(MediaType.APPLICATION_JSON)
public List<Map<Object, Object>> getList(#PathParam("empid")String
emp ) throws Exception {
String id = EmpId.getEmpId(emp);
int employeeid = Integer.parseInt(id);
return reportee.getReportees(employeeid);
}
ReporteeList class:
public class ReporteeList {
public List<Map<Object,Object>> getReportees(int idOfEmp) throws
Exception {
Authentication auth = new Authentication();
JiraCount count = new JiraCount();
String api = "https://*******/core/v3/people/";
int id = idOfEmp;
String ext = "/#reports";
String url = api + id + ext;
String authToken = auth.getToken();
Client restClient = Client.create();
WebResource webResource = restClient.resource(url);
ClientResponse resp =
webResource.accept("application/json").header("Authorization",
"Basic " + authToken)
.get(ClientResponse.class);
if (resp.getStatus() != 200) {
System.err.println("Unable to connect to the server");
}
String output = resp.getEntity(String.class);
// JSONParser reads the data from string object and break each
data into key
// value pairs
JSONParser parse = new JSONParser();
// Type caste the parsed json data in json object
JSONObject jobj = (JSONObject) parse.parse(output);
// Store the JSON object in JSON array as objects (For level 1
array element i.e list)
JSONArray jsonarr_s = (JSONArray) jobj.get("list");
List<Map<Object, Object>> List = new
ArrayList<Map<Object,Object>>();
// Get data for List array
for (int i = 0; i < jsonarr_s.size(); i++) {
Map<Object,Object> map = new HashMap<>();
JSONObject jsonobj_1 = (JSONObject) jsonarr_s.get(i);
JSONObject jive = (JSONObject) jsonobj_1.get("jive");
Object names = jsonobj_1.get("displayName");
Object userid = jive.get("username");
String UserId = userid.toString();
//return the map with the key value pairs
int jiracount = count.getJiraCount(UserId);
map.put("Name", names);
map.put("UserId", userid);
map.put("count", jiracount);
List.add(map);
}
return List;
}
}
Test class:
public class ReporteesListTesting {
ReporteesService Reportee_Service=null;
ReporteeList Reportee_List = mock(ReporteeList.class);
GetEmpId empid = mock(GetEmpId.class);
#Before
public void setup() {
Reportee_Service = new ReporteesService(empid);
}
#Test
public void testReporteeList() throws Exception {
when(Reportee_List.getReportees(54591)).thenReturn("");
assertEquals("",Reportee_Service.getList("vb256121"));
}
}
Now in the Return part I have to return the list, as my getReportees() return the list.
The List contains this data:
"[{UserId=at1234,count=0,Name=Amrith Taj},
{UserId=AR1234,count=1,Name=Anaga R},
{UserId=MS1234,count=4,Name=Madhu S}]"
Please let me know how can this be done,and whether I am on the right track.Please help,I am new to Junits Mockito.
Reportee_Service.getList("vb256121") returns a List of Maps, not a string.
when(Reportee_List.getReportees(54591)).thenReturn(new ArrayList<>());
assertEquals(0 ,Reportee_Service.getList("vb256121").size());
You can do this for a longer query
List<Map<Object, Object>> users = new ArrayList<>();
Map<Object, Object> map = new HashMap<>();
map.put("Name", "Amrith Taj");
map.put("UserId", "at1234");
map.put("count", "0");
users.add(map);
map = new HashMap<>();
map.put("Name", "Anaga R");
map.put("UserId", "AR1234");
map.put("count", "1");
users.add(map);
map = new HashMap<>();
map.put("Name", "Anaga R");
map.put("UserId", "MS1234");
map.put("count", "4");
users.add(map);
when(Reportee_List.getReportees(54591)).thenReturn(users);
assertEquals(3 ,Reportee_Service.getList("vb256121").size());
I want to store the json array sent by php code in java array in android. My php code is working perfectly fine but in the app I get name: as the output. I want to display the names in the texview for checking purpose. Also I want to work with the namesby accessing them one by one.
Php code:
echo json_encode(array("result"=>$result));
Java code:
public class Salary {
public static final String DATA_URL1 = "http://********.com/name.php?salary=";
public static final String KEY_name = "name";
public static final String JSON_ARRAY1 = "result";
}
This is a method of my Name.java
private void showJSON(String response) {
String name = "";
try {
JSONObject jsonObject = new JSONObject(response);
JSONArray result = jsonObject.getJSONArray(Salary.JSON_ARRAY1);
for (int i = 0; i < result.length(); i++) {
JSONObject collegeData = result.getJSONObject(i);
name = collegeData.getString(Salary.KEY_name);
}
} catch (JSONException e) {
e.printStackTrace();
}
textViewResult1.setText("Name:\t" + name);
}
Use GSON Library
ArrayList<Salary> salaryArrayList = new ArrayList<>();
try {
salaryArrayList = new Gson().fromJson(response, new TypeToken<Salary>() {}.getType());
} catch (JsonSyntaxException e) {
e.printStackTrace();
}
Then use salaryArrayList to get values.
Download GSON jar from this link