We Keep Coding

bitcoinJ get transaction value

I downloaded a lot of blockchain data using https://bitcoin.org, I took some file and I try to analyse it with bitcoinj library.
I would like to get information from every transaction:
-who send bitcoins,
-how much,
-who receive bitcoins.
I use:
<dependency>
<groupId>org.bitcoinj</groupId>
<artifactId>bitcoinj-core</artifactId>
<version>0.15.10</version>
</dependency>
I have a code:
NetworkParameters np = new MainNetParams();
Context.getOrCreate(MainNetParams.get());
BlockFileLoader loader = new BlockFileLoader(np,List.of(new File("test/resources/blk00450.dat")));
for (Block block : loader) {
for (Transaction tx : block.getTransactions()) {
System.out.println("Transaction ID" + tx.getTxId().toString());
for (TransactionInput ti : tx.getInputs()) {
// how to get wallet addresses of inputs?
}
// this code works for 99% of transactions but for some throws exceptions
for (TransactionOutput to : tx.getOutputs()) {
// sometimes this line throws: org.bitcoinj.script.ScriptException: Cannot cast this script to an address
System.out.println("out address:" + to.getScriptPubKey().getToAddress(np));
System.out.println("out value:" + to.getValue().toString());
}
}
}
Can you share some snippet that will work for all transactions in the blockchain?

There are at least two type of transaction, P2PKH and P2SH.
Your code would work well with P2PKH, but wouldn not work with P2SH.
You can change the line from:
System.out.println("out address:" + to.getScriptPubKey().getToAddress(np));
to:
System.out.println("out address:" + to.getAddressFromP2PKHScript(np)!=null?to.getAddressFromP2PKHScript(np):to.getAddressFromP2SH(np));
The API of Bitcoin says the methods getAddressFromP2PKHScript() and getAddressFromP2SH() are deprecated, and I have not find suitable method.
However, P2SH means "Pay to Script Hash", which means it could contain two or more public keys to support multi-signature. Moreover, getAddressFromP2SH() returns only one address, perhaps this is the reason why it is deprecated.
I also wrote a convinient method to check the inputs and outputs of a block:
private void printCoinValueInOut(Block block) {
Coin blockInputSum = Coin.ZERO;
Coin blockOutputSum = Coin.ZERO;
System.out.println("--------------------Block["+block.getHashAsString()+"]------"+block.getPrevBlockHash()+"------------------------");
for(Transaction tx : block.getTransactions()) {
Coin txInputSum = tx.getOutputSum();
Coin txOutputSum = tx.getOutputSum();
blockInputSum = blockInputSum.add(txInputSum);
blockOutputSum = blockOutputSum.add(txOutputSum);
System.out.println("Tx["+tx.getTxId()+"]:\t" + txInputSum + "(satoshi) IN, " + txOutputSum + "(satoshi) OUT.");
}
System.out.println("Block total:\t" + blockInputSum + "(satoshi) IN, " + blockOutputSum + "(satoshi) OUT. \n");
}

how to solve “instantiate chaincode” error in fabric-sdk-java?

I use fabrc-sdk-java to operate the e2e_cli network.The e2e uses CA and the TLS is disabled.
I successfully create the channel and install the chaincode.
create channel：
Channel newChannel = client.newChannel(myChannel.getChannelName(), orderer, channelConfiguration, channelConfigurationSignatures.toArray(new byte[myPeerOrgs.size()][]));
channelConfigurationSignatures contains signatures from two organizations.
install chaincode:
Every organization has to send an installation proposal once, using its own peerAdmin organization.
reference：https://github.com/IBM/blockchain-application-using-fabric-java-sdk
But,when I prepare to instantiate chaincode,I get the error:
0endorser failed with Sending proposal to peer0.org1.example.com failed because of: gRPC failure=Status{code=UNKNOWN, description=Failed to deserialize creator identity, err MSP Org1 is unknown, cause=null}. Was verified:false
These are related codes：
client.setUserContext(myPeerOrgs.get(0).getPeerAdmin());
InstantiateProposalRequest instantiateProposalRequest = client.newInstantiationProposalRequest();
instantiateProposalRequest.setProposalWaitTime(fabricConfig.getProposalWaitTime());
instantiateProposalRequest.setChaincodeID(chaincodeID);
instantiateProposalRequest.setFcn(ininFun);
instantiateProposalRequest.setArgs(args);
Map<String, byte[]> tm = new HashMap<>();
tm.put("HyperLedgerFabric", "InstantiateProposalRequest:JavaSDK".getBytes(UTF_8));
tm.put("method", "InstantiateProposalRequest".getBytes(UTF_8));
instantiateProposalRequest.setTransientMap(tm);
ChaincodeEndorsementPolicy chaincodeEndorsementPolicy = new ChaincodeEndorsementPolicy();
chaincodeEndorsementPolicy.fromYamlFile(new File(myChaincode.getChaincodeEndorsementPolicyPath()));
instantiateProposalRequest.setChaincodeEndorsementPolicy(chaincodeEndorsementPolicy);
logger.trace("Sending instantiateProposalRequest to all peers with arguments: " + Arrays.toString(args));
Collection<ProposalResponse> successful = new LinkedList<>();
Collection<ProposalResponse> failed = new LinkedList<>();
Collection<ProposalResponse> responses = channel.sendInstantiationProposal(instantiateProposalRequest);
for (ProposalResponse response : responses) {
if (response.isVerified() && response.getStatus() == ProposalResponse.Status.SUCCESS) {
successful.add(response);
logger.trace(String.format("Succesful instantiate proposal response Txid: %s from peer %s", response.getTransactionID(), response.getPeer().getName()));
} else {
failed.add(response);
}
}
logger.trace(String.format("Received %d instantiate proposal responses. Successful+verified: %d . Failed: %d", responses.size(), successful.size(), failed.size()));
if (failed.size() > 0) {
ProposalResponse first = failed.iterator().next();
logger.error("Not enough endorsers for instantiate :" + successful.size() + "endorser failed with " + first.getMessage() + ". Was verified:" + first.isVerified());
System.exit(1);
}
I thought it was a serialization problem，but the MyUser class and the MyEnrollement class both inherit the Serializable interface, and both define the serialVersionUID.
I have compared blockchain-application-using-fabric-java-sdk and have not identified the problem.

I finally solved this problem.The problem is in the following code:
Channel newChannel = client.newChannel(myChannel.getChannelName(), orderer, channelConfiguration, channelConfigurationSignatures.toArray(new byte[myPeerOrgs.size()][]));
The above code is written by me with reference to End2endIT:
//Create channel that has only one signer that is this orgs peer admin. If channel creation policy needed more signature they would need to be added too.
Channel newChannel = client.newChannel(name, anOrderer, channelConfiguration, client.getChannelConfigurationSignature(channelConfiguration, sampleOrg.getPeerAdmin()));
I don't know if it is wrong with my usage.But my code, the error is in this sentence, when joining the node later, the error is reported.
I referenced https://github.com/IBM/blockchain-application-using-fabric-java-sdk/blob/master/java/src/main/java/org/app/network/CreateChannel.java and found the correct way of writing.
public Channel createChannel() {
logger.info("Begin create channel: " + myChannel.getChannelName());
ChannelConfiguration channelConfiguration = new ChannelConfiguration(new File(fabricConfig.getChannelArtifactsPath() + "/" + myChannel.getChannelName() + ".tx"));
logger.trace("Read channel " + myChannel.getChannelName() + " configuration file：" + fabricConfig.getChannelArtifactsPath() + "/" + myChannel.getChannelName() + ".tx");
byte[] channelConfigurationSignatures = client.getChannelConfigurationSignature(channelConfiguration, myPeerOrgs.get(0).getPeerAdmin());
Channel newChannel = client.newChannel(myChannel.getChannelName(), orderer, channelConfiguration, channelConfigurationSignatures);;
for (Peer peer : myPeerOrgs.get(0).getPeers()) {
// create a channel for the first time, only `joinPeer` here, not `addPeer`
newChannel.joinPeer(peer);
}
for (EventHub eventHub : myPeerOrgs.get(0).getEventHubs()) {
newChannel.addEventHub(eventHub);
}
if (!newChannel.isInitialized()) {
newChannel.initialize();
}
// I have only tested two organizations
// I don’t know if there are any errors in the three organizations.
for (int i = 1; i < myPeerOrgs.size(); i++) {
client.setUserContext(myPeerOrgs.get(i).getPeerAdmin());
newChannel = client.getChannel(myChannel.getChannelName());
for (Peer peer : myPeerOrgs.get(i).getPeers()) {
newChannel.joinPeer(peer);
}
for (EventHub eventHub : myPeerOrgs.get(i).getEventHubs()) {
newChannel.addEventHub(eventHub);
}
}
logger.trace("Node that has joined the channel：");
Collection<Peer> peers = newChannel.getPeers();
for (Peer peer : peers) {
logger.trace(peer.getName() + " at " + peer.getUrl());
}
logger.info("Success, end create channel: " + myChannel.getChannelName() + "\n");
return newChannel;
}
Related code later, such as installing and initializing chaincode, also refer to https://github.com/IBM/blockchain-application-using-fabric-java-sdk. This is an excellent example.
If anyone knows how to use the fourth variable parameter of newChannel, please let me know. Thanks.
Finally, I don't know how to dynamically join nodes, organizations and channels, I am looking for and testing, there are only examples of nodejs on the network, there is no java, if anyone knows, please tell me, I really need. Thanks.

ArrayList in serializable object not carrying over writeUnshared/readUnshared (over network)

I am developing a multiplayer game in Java built around my own client-server architecture. In short, the client requests a copy of the server's World object 30 times a second and, upon receiving it, sets its client-side copy to the response. This is all done using Java's standard net API.
The issue I am having is that I also store an ArrayList of Player objects in the world, and when I add a Player to this list, the client doesn't get the update. It still receives a copy of the world from the server, but its not up to date.
I experienced a similar problem in a past project that was caused by write/readObject and fixed it by using write/readUnshared, but even that isn't working.
Here's the important stuff from the server end of the communication:
String message;
int sum = 0;
while(active)
{
message = "";
try {
message = in.readUTF();
} catch (IOException e) {
active = false;
System.out.println("Lost connection with client " + socket.getInetAddress());
}
if(message.equals("GETWORLD"))
{
try {
sum++;
if(sum == 100)
main.world.addPlayer(999, 2, 2);
System.out.println("Client requested world (#" + sum + ")");
System.out.println(main.world.players.size());
out.writeUnshared(main.world);
out.flush();
System.out.println("Sent client world (#" + sum + ")");
} catch (IOException e) {
active = false;
System.out.println("Lost connection with client " + socket.getInetAddress());
}
}
if(message.equals("DISCONNECT"))
{
active = false;
System.out.println("Client " + socket.getInetAddress() + " requested disconnect");
}
}
And then the client end:
Object read = null;
int sum = 0;
while(active)
{
try {
Thread.sleep((long)(1000 / 30.0));
if(connected)
{
sum++;
System.out.println("Asking server for world (#" + sum + ")");
out.writeUTF("GETWORLD");
out.flush();
read = in.readUnshared();
if(read instanceof World)
{
World temp = (World)read;
System.out.println(temp.players.size());
frame.panel.updateWorld((World)read);
System.out.println("Got world from server (#" + sum + ")");
}
}
} catch (InterruptedException | ClassNotFoundException e1) {
active = false;
e1.printStackTrace();
} catch (IOException e2) {
active = false;
System.out.println("Lost connection with server # " + socket.getInetAddress());
frame.dispose();
System.exit(0);
}
}
Obviously the sum variable is for debugging.
I further tested this with some output, here's what is scaring me:
Client log:
...
Asking server for world (#99)
1
Got world from server (#99)
Asking server for world (#100)
1
Got world from server (#100)
Asking server for world (#101)
1
Got world from server (#101)
...
Server log:
...
Client requested world (#99)
1
Sent client world (#99)
Client requested world (#100)
2
Sent client world (#100)
Client requested world (#101)
2
Sent client world (#101)
...
You can see here that even though the request numbers match up, there's a clear discrepancy between the number of Player objects in the World object.
Here's the important stuff from the World and Player classes for those curious:
public class World implements Serializable
{
public ArrayList<Room> rooms;
public ArrayList<Player> players;
private QuickMaths qm;
...
public class Player implements Serializable
{
private double xPos;
private double yPos;
private Color color;
int id;
...
I apologize if this is a long yet easy problem. I'm not sure if it's a referencing issue or some other network quirk, but it's really driving me nuts. Thanks in advance.

Your problem is with writeUnshared which is a little misleading.
Read here:
"Note that the rules described above only apply to the base-level
object written with writeUnshared, and not to any transitively
referenced sub-objects in the object graph to be serialized. "
This means that the player object will not be written twice but the old reference to that object in the serialization tree will be used.
The solution to this would be to call the reset method after each write call to ensure that the old written objects will not be referenced again.
So:
out.writeUnshared(main.world);
out.flush();
out.reset();

actian JCL example

Does anyone have an example of retrieving data using Actian's JCL to a loosely coupled pervasive database in Java? The database I am connecting to only has DAT files. My goal is to create a link between pervasive and MS SQL.
I am not looking for a freebie, but someone to point me in the right direction so I can learn and grow.
Thank you in advanced!

Found this in my archives. Don't know when it was written, whether it works, or if this interface is still supported. You don't say what version of PSQL you're using so I don't even know if this will work with your version.
import pervasive.database.*;
public class VersionTest implements Consts
{
public VersionTest()
{
try
{
Session session = Driver.establishSession();
Database db = session.connectToDatabase("PMKE:");
XCursor xcursor = db.createXCursor(57000);
//Using local TABL.DAT (length 255 assures no leftovers!)
xcursor.setKZString(0,255,"plsetup\\tabl.dat");
//Open the file to load local MKDE
int status = xcursor.BTRV(BTR_OPEN);
System.out.println("Local Open status: " + status);
//Using remote TABL.DAT (length 255 assures no leftovers!)
xcursor.setKZString(0,255,"h:\\basic2c\\develop\\tabl.dat");
//set the buffer size
xcursor.setDataSize(15);
//get version
status = xcursor.BTRV(BTR_VERSION);
System.out.println("Version status: " + status);
// should be 15, always prints 5
System.out.println("Version length: " + xcursor.getRecLength());
System.out.println("Version: " + xcursor.getDString(0,15));
// try with an open file on a server
XCursor xcursor2 = db.createXCursor(57000);
//Using remote TABL.DAT (length 255 assures no leftovers!)
xcursor2.setKZString(0,255,"h:\\basic2c\\develop\\tabl.dat");
//Open the file
status = xcursor2.BTRV(BTR_OPEN);
System.out.println("Remote Open status: " + status);
//set the buffer size
xcursor2.setDataSize(15);
//get version
status = xcursor2.BTRV(BTR_VERSION);
System.out.println("Version status: " + status);
// should be 15, always prints 5
System.out.println("Version length: " + xcursor2.getRecLength());
System.out.println("Version: " + xcursor2.getDString(0,15));
// clean up resources
Driver.killAllSessions();
}catch(Exception exp)
{
exp.printStackTrace();
}
}
public static void main(String[] args)
{
new VersionTest();
}
}

JCL APIs are still supported with Actian PSQL v12 and v13.
You can find more documentation on retrieving data using Actian JCL at
http://docs.pervasive.com/products/database/psqlv12/wwhelp/wwhimpl/js/html/wwhelp.htm#href=jcl/java_api.2.2.html
To link to MS Sql Server you would need to create the data dictionary files(DDFs) for the PSQl data files to use with relational interfaces.

Random occurrences of java.net.ConnectException

I'm experiencing java.net.ConnectException in random ways.
My servlet runs in Tomcat 6.0 (JDK 1.6).
The servlet periodically fetches data from 4-5 third-party web servers.
The servlet uses a ScheduledExecutorService to fetch the data.
Run locally, all is fine and dandy. Run on my prod server, I see semi-random failures to fetch data from 1 of the third parties (Canadian weather data).
These are the URLs that are failing (plain RSS feeds):
http://weather.gc.ca/rss/city/pe-1_e.xml
http://weather.gc.ca/rss/city/pe-2_e.xml
http://weather.gc.ca/rss/city/pe-3_e.xml
http://weather.gc.ca/rss/city/pe-4_e.xml
http://weather.gc.ca/rss/city/pe-5_e.xml
http://weather.gc.ca/rss/city/pe-6_e.xml
http://meteo.gc.ca/rss/city/pe-1_f.xml
http://meteo.gc.ca/rss/city/pe-2_f.xml
http://meteo.gc.ca/rss/city/pe-3_f.xml
http://meteo.gc.ca/rss/city/pe-4_f.xml
http://meteo.gc.ca/rss/city/pe-5_f.xml
http://meteo.gc.ca/rss/city/pe-6_f.xml
Strange: each cycle, when I periodically fetch this data, the success/fail is all over the map: some succeed, some fail, but it never seems to be the same twice. So, I'm not completely blocked, just randomly blocked.
I slowed down my fetches, by introducing a 61s pause between each one. That had no effect.
The guts of the code that does the actual fetch:
private static final int TIMEOUT = 60*1000; //msecs
public String fetch(String aURL, String aEncoding /*UTF-8*/) {
String result = "";
long start = System.currentTimeMillis();
Scanner scanner = null;
URLConnection connection = null;
try {
URL url = new URL(aURL);
connection = url.openConnection(); //this doesn't talk to the network yet
connection.setConnectTimeout(TIMEOUT);
connection.setReadTimeout(TIMEOUT);
connection.connect(); //actually connects; this shouldn't be needed here
scanner = new Scanner(connection.getInputStream(), aEncoding);
scanner.useDelimiter(END_OF_INPUT);
result = scanner.next();
}
catch (IOException ex) {
long end = System.currentTimeMillis();
long time = end - start;
fLogger.severe(
"Problem connecting to " + aURL + " Encoding:" + aEncoding +
". Exception: " + ex.getMessage() + " " + ex.toString() + " Cause:" + ex.getCause() +
" Connection Timeout: " + connection.getConnectTimeout() + "msecs. Read timeout:" +
connection.getReadTimeout() + "msecs."
+ " Time taken to fail: " + time + " msecs."
);
}
finally {
if (scanner != null) scanner.close();
}
return result;
}
Example log entry showing a failure:
SEVERE: Problem connecting to http://weather.gc.ca/rss/city/pe-5_e.xml Encoding:UTF-8.
Exception: Connection timed out java.net.ConnectException: Connection timed out
Cause:null
Connection Timeout: 60000msecs.
Read timeout:60000msecs.
Time taken to fail: 15028 msecs.
Note that the time to fail is always 15s + a tiny amount.
Also note that it fails to reach the configured 60s timeout for the connection.
The host-server admins (Environment Canada) state that they don't have any kind of a blacklist for the IP address of misbehaving clients.
Also important: the code had been running for several months without this happening.

Someone suggested that instead I should use curl, a bash script, and cron. I implemented that, and it works fine.
I'm not able to solve this problem using Java.