I am getting an error while using broadcast variable inside a CoGroupFunction. The error disappears if appProvider.value() is commented. Have you any idea how to solve this issue ? is the error related to the variable definition or initialization ?
public class UsageJobDS implements Serializable{
private static final Logger log = org.apache.log4j.LogManager.getLogger("myLogger");
Broadcast<Provider> appProvider;
void init(){
// init broadcast variable
....
}
public static void main(String[] args) {
UsageJobDS ujb = new UsageJobDS();
ujb.init();
ujb.run();
}
void run(){
KeyValueGroupedDataset<Long, Row> charges = usageCharges.groupByKey(x -> x.getLong(x.fieldIndex("si__subscription_id")), Encoders.LONG());
Dataset<ProcessEdr> cogg = edrs.cogroup(charges, rateEDRs, Encoders.bean(ProcessEdr.class));
log.warn("Count cogg " + cogg.count());
}
CoGroupFunction<Long, EDR2, Row, ProcessEdr> rateEDRs = (subscription_id, edrsIter, chargesIter) -> {
Logger log = org.apache.log4j.LogManager.getLogger("myLogger");
log.warn("inside rateEDRs function");
while (edrsIter.hasNext()) {
appProvider.value(); // HERE
}
return results.iterator();
};
}
and I am getting this error
java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.opencell.spark.jobs.UsageJobDS.rateEDRs of type org.apache.spark.api.java.function.CoGroupFunction in instance of org.opencell.spark.jobs.UsageJobDS
at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2233)
at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1405)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2288)
Actually it works if the cogroup function definition is changed to the below code. However, the error cause still unknown.
Dataset<ProcessEdr> cogg = edrs.cogroup(charges, (subscription_id, edrsIter, chargesIter) -> {
ArrayList<ProcessEdr> results = new ArrayList<>();
System.out.println("App Provider name" + appProvider.value().getIssuer_name());
return results.iterator();
}, Encoders.bean(ProcessEdr.class));
Related
I am trying to take data from one class, put it in a singleton, and then proceed to access that data in another class.
Here is my singleton:
public class DataTransferrer {
private static DataTransferrer instance = null;
private ArrayList<Transients> transientList;
private DataTransferrer(){
transientList = new ArrayList<>();
}
public static synchronized DataTransferrer getInstance(){
if(instance == null) {instance = new DataTransferrer();}
return instance;
}
public ArrayList<Transients> getTransients(){return this.transientList;}
public void setTransients(ArrayList<Transients> t){transientList = t;}
public void printAll(){
for(Transients t : transientList){
Log.d("DEBUG DA",t.getDa().toString());
}
}
}
Here is my method that adds data to an arrayList which is then set to the arrayList within the singleton to then be used elsewhere:
public class JSONParser {
// URL to get contacts JSON
private static String url = "http://pi.cs.oswego.edu/~lpatmore/getAllTransients.php";
ArrayList<Transients> transientList;
private Transients t;
public JSONParser(){
transientList = new ArrayList<>();
}
public void execute(){
new GetTransients().execute();
}
/**
* Async task class to get json by making HTTP call
*/
private class GetTransients extends AsyncTask<Void, Void, Void> {
protected Void doInBackground(Void... arg0) {
HttpHandler sh = new HttpHandler();
// Making a request to url and getting response
String jsonStr = sh.makeServiceCall(url);
Log.d(TAG, "Response from url: " + jsonStr);
if (jsonStr != null) {
try {
Log.d(TAG, "NOT NULL DEBUG");
JSONObject jsonObj = new JSONObject(jsonStr);
// Getting JSON Array node
JSONArray transients = jsonObj.getJSONArray("result");
// looping through All Transients
for (int i = 0; i < transients.length(); i++) {
JSONObject c = transients.getJSONObject(i);
String author = c.getString("author");
String transientId = c.getString("transientId");
String dateAlerted = c.getString("dateAlerted");
String datePublished = c.getString("datePublished");
float right_asencsion = (float) c.getDouble("right_asencsion");
float declination = (float) c.getDouble("declination");
t = new Transients(author, transientId, dateAlerted, datePublished, right_asencsion, declination);
// adding trans to trans list
transientList.add(t);
}
DataTransferrer.getInstance().setTransients(transientList);
DataTransferrer.getInstance().printAll();
} catch (final JSONException e) {
Log.e(TAG, "Json parsing error: " + e.getMessage());
}
} else {
Log.e(TAG, "Couldn't get json from server.");
}
return null;
}
}
}
And I try to access it in another class like:
for(int i = 0; i < DataTransferrer.getInstance().getTransients().size(); i++){
System.out.println("DEBUG ADDED TRANSIENT");
Float ra = DataTransferrer.getInstance().getTransients().get(i).getR();
Float dec = DataTransferrer.getInstance().getTransients().get(i).getD();
transients.add(new Transient("Transient", names[i], GeocentricCoordinates.getInstance(ra, dec)));
}
Whenever I debug it checking to see in the other class, I am trying to access the arraylist, but in the other class it's like there was never an array list added to getTransients() method.
I tested this exact setup in Eclipse and it works perfectly but not in Android Studio.
I tested this exact setup in Eclipse and it works perfectly but not in
Android Studio.
Basically, you have got multithreading issue i.e., your DataTransferrer class is NOT threadsafe which is causing the issue i.e., there is a race condition between multiple threads and your singleton instance is NOT actually singleton (multiple objects are being created for DataTransferrer class).
So when you calling setTransients() from one thread, it is actually storing the Transients in one instance of DataTransferrer and then when you are trying retrieve the Transients from a different thread it is using different instance (object) of DataTransferrer.
You can create threadsafe singleton instance for your DataTransferrer class as shown below:
public class DataTransferrer {
private static final DataTransferrer instance = new DataTransferrer();
private ArrayList<Transients> transientList = new ArrayList<>();
//private constructor, so instantiation NOT allowed
private DataTransferrer() {}
public static synchronized DataTransferrer getInstance(){
return instance;
}
public ArrayList<Transients> getTransients(){
return this.transientList;
}
public void setTransients(ArrayList<Transients> t){
transientList = t;
}
public void printAll(){
for(Transients t : transientList){
Log.d("DEBUG DA",t.getDa().toString());
}
}
}
Also, note that in the above code, the instance will be created eagerly during the startup (initializing) time and when you call setTransients and getTransients that same instance (singleton) will be used. Instead of eager creation, if you need to create the instance lazily, you can look here.
I need to configure some attribute in ScriptEngine- or ScriptContext-level, to be used in Java methods.
So, how to get a reference to that ScriptContext in order to retrieve the value?
Example: setting the attribute:
public static void main(String[] args) throws Exception {
ScriptEngine engine = new ScriptEngineManager().getEngineByName("nashorn");
engine.getContext().setAttribute("param1", "paramValue", ScriptContext.ENGINE_SCOPE);
engine.put("MyWindow", engine.eval("Java.type(\"" + MyWindow.class.getName() + "\")"));
engine.eval("print(new MyWindow().test());");
}
MyWindow implementation: how to get that attribute?
public class MyWindow {
public String test() {
// how to get 'param1' value here
return "in test";
}
}
Pass it in:
engine.eval("print(new MyWindow().test(param1));");
// ^^^^^^
// vvvvvvvvvvvvv
public String test(String param1) {
// how to get 'param1' value here
return "in test";
}
Update
If you have code with a call stack like javaMethod1 -> JavaScript -> javaMethod2, and you want a value from javaMethod1 to be available to javaMethod2, without changing the JavaScript to pass it on, use a ThreadLocal.
Since your code is in main you could just use a static directly, but I'm assuming your context is more complex. The code below works even in multi-threaded contexts. The ThreadLocal can be stored anywhere, it just has to be static and available to both java methods.
public static void main(String[] args) throws Exception {
MyWindow.param1.set("paramValue");
try {
ScriptEngine engine = new ScriptEngineManager().getEngineByName("nashorn");
engine.eval("var MyWindow = Java.type(\"" + MyWindow.class.getName() + "\");" +
"print(new MyWindow().test());");
} finally {
MyWindow.param1.remove();
}
}
public class MyWindow {
public static final ThreadLocal<String> param1 = new ThreadLocal<>();
public String test() {
String value = param1.get();
return "in test: param1 = " + value;
}
}
Can a package final variable be changed through reflection?
Say I have this:
public class Widget {
final int val = 23;
}
Can the val be changed through reflection if made accessible?
If so, is there any way to prevent it that without using the security manager?
YES. Try this code:
public static void main(String[] args) throws Exception {
Widget w = new Widget ();
Field m = Widget.class.getDeclaredField("val");
m.setAccessible(true);
m.set(w, 233);
System.out.println(m.get(w)); /// PRINT 233
}
Turns out, changing final members causes reflection-obtained values to differ from values returned by regular code! This is quite scary.
import java.lang.reflect.Field;
public class Test {
private static final class Widget {
private final int val = 23;
public int getVal() {
return val;
}
}
public static void main(String[] args) throws Exception {
Widget w = new Widget ();
Field m = Widget.class.getDeclaredField("val");
m.setAccessible(true);
m.set(w, 233);
Field m1 = Widget.class.getDeclaredField("val");
m1.setAccessible(true);
System.out.println(m.get(w)); /// PRINT 233
System.out.println(w.getVal()); /// PRINT 23
System.out.println(m1.get(w)); /// PRINT 233
}
}
Try this.
Widget() {
checkPerMission();
}
private void checkPerMission() {
Class self = sun.reflect.Reflection.getCallerClass(1);
Class caller = sun.reflect.Reflection.getCallerClass(3);
if (self != caller) {
throw new java.lang.IllegalAccessError();
}
}
I am trying to understand the usage of Java extend.... I created a sample testing code to under how it works....
public class Parent {
public String Msg="Original";
public String getMsg() {
return Msg;
}
public void setMsg(String msg) {
Msg = msg;
}
public void printing(){
System.out.println(Msg);
}
}
public class Child extends Parent{
public HashMap<String, String> Msg2;
public Integer Msg3;
public HashMap<String, String> getMsg2() {
return Msg2;
}
public void setMsg2(HashMap<String, String> msg2) {
Msg2 = msg2;
}
public void printing(){
System.out.println("1 : " + Msg);
System.out.println( Msg2 );
}
public static void main(String[] args) {
Child a = new Child();
System.out.println(a.Msg.getClass()); // able to detect variable from parent
System.out.println(a.Msg2.getClass()); // Not able to detected, even variable from
// same instance object child
System.out.println(a.Msg3.getClass()); // Not able to detected, even variable from
// same instance object child
a.printing();
}
}
I getting confuse why Msg variable from parent object can detected easy.
While Msg2 and Msg 3 coming from the same instance Child -> a can't recognize it's own variable.
The error message getting from Msg2 or Msg 3 is, Exception in thread "main" java.lang.NullPointerException
Is there anyone able to explain why java behave in such way ?
Thank you....
You have not initialized Msg2 and Msg3. You need to use the new keyword to initialize them, so that they are not null.
add these two statements.
Msg2 = new HashMap<String,String>();
Msg3 = new Integer();
Because Msg has been initialised to a non-null value (the string "Original") but the other fields have not, so they're null and you get an exception trying to call a method (getClass) on a null reference. If you just tried to print out Msg2 rather than Msg2.getClass() then you'd see the value null with no exception.
first set a value to your msg in your child class so its not null by default.
Int Msg3 = 5;
and display it this way
System.out.println("Here is a msg:" + child.Msg4);
Not sure if the title makes sense, but I am trying to return a Success message from a class that receives a linkedhashmap, however eclipse is giving me error when I try to compile the files, offering
Remove arguments to match 'logFile()'
Create constructor 'logFile(Map<String, String>)'
How do set it up to send a Map and revieve a String?
thx
Art
Code corrected as per #Jeff Storey below with error suppression for eclipse
calling class
eventLog.put(stringA,stringB);
logFile logStuff = new logFile();
successRtn = logFile.Process(eventLog);
// Do Stuff with SuccessRtn
logFile class
public class logFile {
static String Success = "Fail";
public static String Process(Map<String, String> eventlog){
// Do Stuff
Success = "Yeh!"
return Success;
}
public static void main(String[] args){
#SuppressWarnings("static-access")
String result = new logFile().Procces(eventLog);
System.out.println("result = " + result);
}
The main method is a special method whose signature must public static void main(String[] args) when being used as an entry point to your application. Create a second method that does the actual work, like this:
public class LogFile {
public String process(Map<String,String> eventLog) {
// do stuff
return success;
}
public void main(String[] args) {
// eventLog will probably be read from a filepath passed into the args
String result = new LogFile().process(eventLog);
System.out.println("result = " + result);
}
}
Note that a lot of your naming conventions are also non standard. Classes should begin with a capital letter and variables should begin with a lower case.