Reading JSON from pages incrementally - java

I have a java client that reads JSON from a web service. The first part of thejson response (offset 1 - 20) is of the format below. I can currently parse the first page of the data into a file (though i will love to parse into db eventually). How can I get the client to parse the next page (offset 21 - 40) ? The second part of the file has more information intentionally omitted for brevity sake. I have used the apache http client but it seems there is no method to handle this ? Will be grateful if pointed to the right directoin.
{
"meta": {
"limit": 15,
"next": "/abc/api/defapi/lkk/?username=abc&api_key=xyz&limit=15&offset=60&format=json",
"offset": 30,
"previous": "/abc/api/defapi/lkk/?username=abc&api_key=xyz&limit=15&offset=30&format=json",
"total_count": 312213
},
"objects": [
{
"availability": "COMPLETE",
"confidentiality": "",
"cpe_id": [
{
"name": "cpe:/o:sun:sunos:5.11::express"
}
NOTE:truncated for brevity

Many thanks all the answers and pointers to my question. The code below does what I wanted i.e. grabs json files and save on disk, incrementally. I think it is still very basic hence I would appreciate all criticisms on better approaches, coding styles. Still learning !!
while (count < totalCount ) {
MyClient check = new MyClient();
check.checkSecurity();
System.out.println(" ************ Starting with count " + count + " ***********************");
URI uriBuilder = new URIBuilder(url)
.addParameter("limit", String.valueOf(limit))
.addParameter("offset", String.valueOf(offset))
.build();
System.out.println("The URL built is --> " + uriBuilder);
HttpGet hget = new HttpGet(uriBuilder);
System.out.println(hget.getRequestLine());
CloseableHttpClient httpclient = HttpClients.createDefault();
hget.addHeader("Content-Type", "application/json");
HttpResponse response = httpclient.execute(hget);
URL vdbUrl = uriBuilder.toURL();
System.out.println("Response status : " + response.getStatusLine());
HttpEntity entity2 = response.getEntity();
do {
vdbInput = new File("C:/myfiles/entry_0" + filenamePostfix + ".json");
FileUtils.copyURLToFile(vdbUrl, vdbInput);
System.out.println("File successfully written from page " + count );
System.out.println("page is : " + vdbUrl);
filenamePostfix++;
count++;
offset = offset + 20;
} while (filenamePostfix < count) ;
EntityUtils.consume(entity2);
}
}

Related

how to solve “instantiate chaincode” error in fabric-sdk-java?

I use fabrc-sdk-java to operate the e2e_cli network.The e2e uses CA and the TLS is disabled.
I successfully create the channel and install the chaincode.
create channel:
Channel newChannel = client.newChannel(myChannel.getChannelName(), orderer, channelConfiguration, channelConfigurationSignatures.toArray(new byte[myPeerOrgs.size()][]));
channelConfigurationSignatures contains signatures from two organizations.
install chaincode:
Every organization has to send an installation proposal once, using its own peerAdmin organization.
reference:https://github.com/IBM/blockchain-application-using-fabric-java-sdk
But,when I prepare to instantiate chaincode,I get the error:
0endorser failed with Sending proposal to peer0.org1.example.com failed because of: gRPC failure=Status{code=UNKNOWN, description=Failed to deserialize creator identity, err MSP Org1 is unknown, cause=null}. Was verified:false
These are related codes:
client.setUserContext(myPeerOrgs.get(0).getPeerAdmin());
InstantiateProposalRequest instantiateProposalRequest = client.newInstantiationProposalRequest();
instantiateProposalRequest.setProposalWaitTime(fabricConfig.getProposalWaitTime());
instantiateProposalRequest.setChaincodeID(chaincodeID);
instantiateProposalRequest.setFcn(ininFun);
instantiateProposalRequest.setArgs(args);
Map<String, byte[]> tm = new HashMap<>();
tm.put("HyperLedgerFabric", "InstantiateProposalRequest:JavaSDK".getBytes(UTF_8));
tm.put("method", "InstantiateProposalRequest".getBytes(UTF_8));
instantiateProposalRequest.setTransientMap(tm);
ChaincodeEndorsementPolicy chaincodeEndorsementPolicy = new ChaincodeEndorsementPolicy();
chaincodeEndorsementPolicy.fromYamlFile(new File(myChaincode.getChaincodeEndorsementPolicyPath()));
instantiateProposalRequest.setChaincodeEndorsementPolicy(chaincodeEndorsementPolicy);
logger.trace("Sending instantiateProposalRequest to all peers with arguments: " + Arrays.toString(args));
Collection<ProposalResponse> successful = new LinkedList<>();
Collection<ProposalResponse> failed = new LinkedList<>();
Collection<ProposalResponse> responses = channel.sendInstantiationProposal(instantiateProposalRequest);
for (ProposalResponse response : responses) {
if (response.isVerified() && response.getStatus() == ProposalResponse.Status.SUCCESS) {
successful.add(response);
logger.trace(String.format("Succesful instantiate proposal response Txid: %s from peer %s", response.getTransactionID(), response.getPeer().getName()));
} else {
failed.add(response);
}
}
logger.trace(String.format("Received %d instantiate proposal responses. Successful+verified: %d . Failed: %d", responses.size(), successful.size(), failed.size()));
if (failed.size() > 0) {
ProposalResponse first = failed.iterator().next();
logger.error("Not enough endorsers for instantiate :" + successful.size() + "endorser failed with " + first.getMessage() + ". Was verified:" + first.isVerified());
System.exit(1);
}
I thought it was a serialization problem,but the MyUser class and the MyEnrollement class both inherit the Serializable interface, and both define the serialVersionUID.
I have compared blockchain-application-using-fabric-java-sdk and have not identified the problem.
I finally solved this problem.The problem is in the following code:
Channel newChannel = client.newChannel(myChannel.getChannelName(), orderer, channelConfiguration, channelConfigurationSignatures.toArray(new byte[myPeerOrgs.size()][]));
The above code is written by me with reference to End2endIT:
//Create channel that has only one signer that is this orgs peer admin. If channel creation policy needed more signature they would need to be added too.
Channel newChannel = client.newChannel(name, anOrderer, channelConfiguration, client.getChannelConfigurationSignature(channelConfiguration, sampleOrg.getPeerAdmin()));
I don't know if it is wrong with my usage.But my code, the error is in this sentence, when joining the node later, the error is reported.
I referenced https://github.com/IBM/blockchain-application-using-fabric-java-sdk/blob/master/java/src/main/java/org/app/network/CreateChannel.java and found the correct way of writing.
public Channel createChannel() {
logger.info("Begin create channel: " + myChannel.getChannelName());
ChannelConfiguration channelConfiguration = new ChannelConfiguration(new File(fabricConfig.getChannelArtifactsPath() + "/" + myChannel.getChannelName() + ".tx"));
logger.trace("Read channel " + myChannel.getChannelName() + " configuration file:" + fabricConfig.getChannelArtifactsPath() + "/" + myChannel.getChannelName() + ".tx");
byte[] channelConfigurationSignatures = client.getChannelConfigurationSignature(channelConfiguration, myPeerOrgs.get(0).getPeerAdmin());
Channel newChannel = client.newChannel(myChannel.getChannelName(), orderer, channelConfiguration, channelConfigurationSignatures);;
for (Peer peer : myPeerOrgs.get(0).getPeers()) {
// create a channel for the first time, only `joinPeer` here, not `addPeer`
newChannel.joinPeer(peer);
}
for (EventHub eventHub : myPeerOrgs.get(0).getEventHubs()) {
newChannel.addEventHub(eventHub);
}
if (!newChannel.isInitialized()) {
newChannel.initialize();
}
// I have only tested two organizations
// I don’t know if there are any errors in the three organizations.
for (int i = 1; i < myPeerOrgs.size(); i++) {
client.setUserContext(myPeerOrgs.get(i).getPeerAdmin());
newChannel = client.getChannel(myChannel.getChannelName());
for (Peer peer : myPeerOrgs.get(i).getPeers()) {
newChannel.joinPeer(peer);
}
for (EventHub eventHub : myPeerOrgs.get(i).getEventHubs()) {
newChannel.addEventHub(eventHub);
}
}
logger.trace("Node that has joined the channel:");
Collection<Peer> peers = newChannel.getPeers();
for (Peer peer : peers) {
logger.trace(peer.getName() + " at " + peer.getUrl());
}
logger.info("Success, end create channel: " + myChannel.getChannelName() + "\n");
return newChannel;
}
Related code later, such as installing and initializing chaincode, also refer to https://github.com/IBM/blockchain-application-using-fabric-java-sdk. This is an excellent example.
If anyone knows how to use the fourth variable parameter of newChannel, please let me know. Thanks.
Finally, I don't know how to dynamically join nodes, organizations and channels, I am looking for and testing, there are only examples of nodejs on the network, there is no java, if anyone knows, please tell me, I really need. Thanks.

youtube.subscriptions.list (api v3) - nextPageToken isn't available

I'm trying to get all channels from my subscriptions. But the "nextPageToken" isn't available.
The response should containing "nextPageToken":
(from developers.google.com - YouTube (v3) - Subscriptions: list)
{
"kind": "youtube#subscriptionListResponse",
"etag": etag,
"nextPageToken": string,
"prevPageToken": string,
"pageInfo": {
"totalResults": integer,
"resultsPerPage": integer
},
"items": [
subscription Resource
]
}
This is my request:
GET https://www.googleapis.com/youtube/v3/subscriptions?part=snippet&maxResults=10&mine=true&key={YOUR_API_KEY}
APIs Explorer - YouTube (v3) - Subscriptions.list:
https://developers.google.com/apis-explorer/#p/youtube/v3/youtube.subscriptions.list?part=snippet&maxResults=10&mine=true
My response:
{
"kind": "youtube#subscriptionListResponse",
"etag": "\"XXXXX/XXXXX\"",
"pageInfo": {
"totalResults": 115,
"resultsPerPage": 10
},
"items": [
...
Can you tell me why the nextPageToken is missing, please?
I have now a workaround for this.
Please tell me if that helps.
The tokens seems to be the same for each page of other API Youtube V3 API calls, so I can use it to fetch all pages of subscriptions I need.
tokens = ['CDIQAA','CGQQAA','CJYBEAA','CMgBEAA','CPoBEAA','CKwCEAA','CN4CEAA','CJADEAA','CMIDEAA','CPQDEAA','CKYEEAA', ...]
You can use ANOTHER Youtube API to get more page tokens if you need more. Just fetch 1 element a time and log the tokens to use in this API.
I just need to known when to stop... so I checked when API calls returned no channels!
#retry(stop_max_attempt_number=7)
def get_subscription_page(self, channel_id, pageToken):
print 'Retrieving subscription page using Youtube API (token: %s)' % pageToken
res = self.youtube_data_api.subscriptions().list(part="id,snippet,contentDetails",channelId=channel_id, maxResults=50, pageToken=pageToken).execute()
return res
def get_subscriptions(self, channel_id):
self.authorize(channel_id)
subs = []
# Tokens to deal with api bug...
# https://code.google.com/p/gdata-issues/issues/detail?id=7163
tokens = ['CDIQAA','CGQQAA','CJYBEAA','CMgBEAA','CPoBEAA','CKwCEAA','CN4CEAA','CJADEAA','CMIDEAA','CPQDEAA','CKYEEAA']
iPage = 0
pageToken = ''
while True:
res = self.get_subscription_page(channel_id, pageToken)
channelIds = []
for channel in res['items']: channelIds.append(channel.get('snippet').get('resourceId').get('channelId'))
pageToken = res.get('nextPageToken')
# If no next page token is returned... it might be caused by a bug.
# This workaroud will only have effect when the bug still lives.
if not pageToken:
if not channelIds:
# Workaroud for this: https://code.google.com/p/gdata-issues/issues/detail?id=7163
print ' (Workaround due to API bug) No channels returned in this API call! Finished!'
break
else:
pageToken = tokens[iPage]
# get channel info for each channel ID
channelsInfo = self.get_channel_info(channelIds)
subs += channelsInfo
print ' Itens already retrieved: %d ' % len(subs)
iPage += 1
if args.debug: break
if pageToken: continue
print 'No more pages to retrieve!'
break
return subs
Here is a JS snippet I came up with to generate pageTokens up to at least 1024, I cannot guarantee that it will produce anything valid beyond that as i could not find any service which will get me tokens for offsets > 450 to validate my guesses and assumptions.
var d0 = "AEIMQUYcgkosw048";
var d1 = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
var d2 = d1;
var d1c = 0;
var d2c = 0;
var overflowSuffix = "Q";
var direction = "AA";
var d2OverflowCounter = 0;
var pageSize = 50;
for (i = 0; i < 1024; i++) {
if (i % pageSize == 0) console.log("C" + d1.charAt((d1c / d0.length) % d1.length) + d0.charAt(i % d0.length) + overflowSuffix + direction, ":", i);
if (++d1c % (1 << 8) == 0) d1c = 1 << 7;
if (++d2c % (1 << 7) == 0) overflowSuffix = d2.charAt(++d2OverflowCounter) + "E";
}
(check developer tools / console to see generated codes)
I have a script that runs each hour based on this Youtube API V3 and it stopped to work 4 hours ago. The nextPageToken is not available anymore. Before, it was available exactly like in the first code you posted.

Viewing data on browser

i have a domain with fields like URL, lastModified etc. Now i want to view the information i get from this code into the list page. How can i do tht
def url = new URL("http://www.google.com")
HttpURLConnection connection = (HttpURLConnection)url.openConnection()
// long contentLength = Long.parseLong(connection.getHeaderField("Content-Length"));
connection.setRequestMethod("GET")
connection.connect()
connection.getURL()
println("Date: " + new Date(connection.getDate()))
println("LastModified Date:" + new Date(connection.getLastModified()))
if (connection.responseCode == 200 || connection.responseCode == 201){
def returnMessage = connection.content
//print out the full response
println "${returnMessage}";
System.out.println(connection.getURL());
} else {
println "Error Connecting to " + url
println "Couldn't connect to url"
}
From your controller, you can use the render method,
render returnMessage
(see http://grails.org/doc/2.0.x/ref/Controllers/render.html)
or pass it in a model to a gsp page and then print it from gsp,
return [returnMsg: returnMessage]
....(in .gsp)
${returnMsg}
(see http://grails.org/doc/2.0.x/guide/theWebLayer.html#GSPBasics)

Upload a video file by chunks

Yes, it's a long question with a lot of detail... So, my question is: How can I stream an upload to Vimeo in segments?
For anyone wanting to copy and debug on their own machine: Here are the things you need:
My code here.
Include the Scribe library found here
Have a valid video file (mp4) which is at least greater than 10 MB and put it in the directory C:\test.mp4 or change that code to point wherever yours is.
That's it! Thanks for helping me out!
Big update: I've left a working API Key and Secret for Vimeo in the code here. So as long as you have a Vimeo account, all the code should work just fine for you once you've allowed the application and entered your token. Just copy the code from that link into a project on your favorite IDE and see if you can fix this with me. I'll give the bounty to whoever gives me the working code. Thanks! Oh, and don't expect to use this Key and Secret for long. Once this problem's resolved I'll delete it. :)
Overview of the problem: The problem is when I send the last chunk of bytes to Vimeo and then verify the upload, the response returns that the length of all the content is the length of only the last chunk, not all the chunks combined as it should be.
SSCCE Note: I have my entire SSCCE here. I put it somewhere else so it can be C ompilable. It is NOT very S hort (about 300 lines), but hopefully you find it to be S elf-contained, and it's certainly an E xample!). I am, however, posting the relevant portions of my code in this post.
This is how it works: When you upload a video to Vimeo via the streaming method (see Upload API documentation here for setup to get to this point), you have to give a few headers: endpoint, content-length, and content-type. The documentation says it ignores any other headers. You also give it a payload of the byte information for the file you're uploading. And then sign and send it (I have a method which will do this using scribe).
My problem: Everything works great when I just send the video in one request. My problem is in cases when I'm uploading several bigger files, the computer I'm using doesn't have enough memory to load all of that byte information and put it in the HTTP PUT request, so I have to split it up into 1 MB segments. This is where things get tricky. The documentation mentions that it's possible to "resume" uploads, so I'm trying to do that with my code, but it's not working quite right. Below, you'll see the code for sending the video. Remember my SSCCE is here.
Things I've tried: I'm thinking it has something to do with the Content-Range header... So here are the things I've tried in changing what the Content-Range header says...
Not adding content range header to the first chunk
Adding a prefix to the content range header (each with a combination of the previous header):
"bytes"
"bytes " (throws connection error, see the very bottom for the error) --> It appears in the documentation that this is what they're looking for, but I'm pretty sure there are typos in the documentation because they have the content-range header on their "resume" example as: 1001-339108/339108 when it should be 1001-339107/339108. So... Yeah...
"bytes%20"
"bytes:"
"bytes: "
"bytes="
"bytes= "
Not adding anything as a prefix to the content range header
Here's the code:
/**
* Send the video data
*
* #return whether the video successfully sent
*/
private static boolean sendVideo(String endpoint, File file) throws FileNotFoundException, IOException {
// Setup File
long contentLength = file.length();
String contentLengthString = Long.toString(contentLength);
FileInputStream is = new FileInputStream(file);
int bufferSize = 10485760; // 10 MB = 10485760 bytes
byte[] bytesPortion = new byte[bufferSize];
int byteNumber = 0;
int maxAttempts = 1;
while (is.read(bytesPortion, 0, bufferSize) != -1) {
String contentRange = Integer.toString(byteNumber);
long bytesLeft = contentLength - byteNumber;
System.out.println(newline + newline + "Bytes Left: " + bytesLeft);
if (bytesLeft < bufferSize) {
//copy the bytesPortion array into a smaller array containing only the remaining bytes
bytesPortion = Arrays.copyOf(bytesPortion, (int) bytesLeft);
//This just makes it so it doesn't throw an IndexOutOfBounds exception on the next while iteration. It shouldn't get past another iteration
bufferSize = (int) bytesLeft;
}
byteNumber += bytesPortion.length;
contentRange += "-" + (byteNumber - 1) + "/" + contentLengthString;
int attempts = 0;
boolean success = false;
while (attempts < maxAttempts && !success) {
int bytesOnServer = sendVideoBytes("Test video", endpoint, contentLengthString, "video/mp4", contentRange, bytesPortion, first);
if (bytesOnServer == byteNumber) {
success = true;
} else {
System.out.println(bytesOnServer + " != " + byteNumber);
System.out.println("Success is not true!");
}
attempts++;
}
first = true;
if (!success) {
return false;
}
}
return true;
}
/**
* Sends the given bytes to the given endpoint
*
* #return the last byte on the server (from verifyUpload(endpoint))
*/
private static int sendVideoBytes(String videoTitle, String endpoint, String contentLength, String fileType, String contentRange, byte[] fileBytes, boolean addContentRange) throws FileNotFoundException, IOException {
OAuthRequest request = new OAuthRequest(Verb.PUT, endpoint);
request.addHeader("Content-Length", contentLength);
request.addHeader("Content-Type", fileType);
if (addContentRange) {
request.addHeader("Content-Range", contentRangeHeaderPrefix + contentRange);
}
request.addPayload(fileBytes);
Response response = signAndSendToVimeo(request, "sendVideo on " + videoTitle, false);
if (response.getCode() != 200 && !response.isSuccessful()) {
return -1;
}
return verifyUpload(endpoint);
}
/**
* Verifies the upload and returns whether it's successful
*
* #param endpoint to verify upload to
* #return the last byte on the server
*/
public static int verifyUpload(String endpoint) {
// Verify the upload
OAuthRequest request = new OAuthRequest(Verb.PUT, endpoint);
request.addHeader("Content-Length", "0");
request.addHeader("Content-Range", "bytes */*");
Response response = signAndSendToVimeo(request, "verifyUpload to " + endpoint, true);
if (response.getCode() != 308 || !response.isSuccessful()) {
return -1;
}
String range = response.getHeader("Range");
//range = "bytes=0-10485759"
return Integer.parseInt(range.substring(range.lastIndexOf("-") + 1)) + 1;
//The + 1 at the end is because Vimeo gives you 0-whatever byte where 0 = the first byte
}
Here's the signAndSendToVimeo method:
/**
* Signs the request and sends it. Returns the response.
*
* #param service
* #param accessToken
* #param request
* #return response
*/
public static Response signAndSendToVimeo(OAuthRequest request, String description, boolean printBody) throws org.scribe.exceptions.OAuthException {
System.out.println(newline + newline
+ "Signing " + description + " request:"
+ ((printBody && !request.getBodyContents().isEmpty()) ? newline + "\tBody Contents:" + request.getBodyContents() : "")
+ ((!request.getHeaders().isEmpty()) ? newline + "\tHeaders: " + request.getHeaders() : ""));
service.signRequest(accessToken, request);
printRequest(request, description);
Response response = request.send();
printResponse(response, description, printBody);
return response;
}
And here's some (an example... All of the output can be found here) of the output from the printRequest and printResponse methods: NOTE This output changes depending on what the contentRangeHeaderPrefix is set to and the first boolean is set to (which specifies whether or not to include the Content-Range header on the first chunk).
We're sending the video for upload!
Bytes Left: 15125120
Signing sendVideo on Test video request:
Headers: {Content-Length=15125120, Content-Type=video/mp4, Content-Range=bytes%200-10485759/15125120}
sendVideo on Test video >>> Request
Headers: {Authorization=OAuth oauth_signature="zUdkaaoJyvz%2Bt6zoMvAFvX0DRkc%3D", oauth_version="1.0", oauth_nonce="340477132", oauth_signature_method="HMAC-SHA1", oauth_consumer_key="5cb447d1fc4c3308e2c6531e45bcadf1", oauth_token="460633205c55d3f1806bcab04174ae09", oauth_timestamp="1334336004", Content-Length=15125120, Content-Type=video/mp4, Content-Range=bytes: 0-10485759/15125120}
Verb: PUT
Complete URL: http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d
sendVideo on Test video >>> Response
Code: 200
Headers: {null=HTTP/1.1 200 OK, Content-Length=0, Connection=close, Content-Type=text/plain, Server=Vimeo/1.0}
Signing verifyUpload to http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d request:
Headers: {Content-Length=0, Content-Range=bytes */*}
verifyUpload to http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d >>> Request
Headers: {Authorization=OAuth oauth_signature="FQg8HJe84nrUTdyvMJGM37dpNpI%3D", oauth_version="1.0", oauth_nonce="298157825", oauth_signature_method="HMAC-SHA1", oauth_consumer_key="5cb447d1fc4c3308e2c6531e45bcadf1", oauth_token="460633205c55d3f1806bcab04174ae09", oauth_timestamp="1334336015", Content-Length=0, Content-Range=bytes */*}
Verb: PUT
Complete URL: http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d
verifyUpload to http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d >>> Response
Code: 308
Headers: {null=HTTP/1.1 308 Resume Incomplete, Range=bytes=0-10485759, Content-Length=0, Connection=close, Content-Type=text/plain, Server=Vimeo/1.0}
Body:
Bytes Left: 4639360
Signing sendVideo on Test video request:
Headers: {Content-Length=15125120, Content-Type=video/mp4, Content-Range=bytes: 10485760-15125119/15125120}
sendVideo on Test video >>> Request
Headers: {Authorization=OAuth oauth_signature="qspQBu42HVhQ7sDpzKGeu3%2Bn8tM%3D", oauth_version="1.0", oauth_nonce="183131870", oauth_signature_method="HMAC-SHA1", oauth_consumer_key="5cb447d1fc4c3308e2c6531e45bcadf1", oauth_token="460633205c55d3f1806bcab04174ae09", oauth_timestamp="1334336015", Content-Length=15125120, Content-Type=video/mp4, Content-Range=bytes%2010485760-15125119/15125120}
Verb: PUT
Complete URL: http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d
sendVideo on Test video >>> Response
Code: 200
Headers: {null=HTTP/1.1 200 OK, Content-Length=0, Connection=close, Content-Type=text/plain, Server=Vimeo/1.0}
Signing verifyUpload to http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d request:
Headers: {Content-Length=0, Content-Range=bytes */*}
verifyUpload to http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d >>> Request
Headers: {Authorization=OAuth oauth_signature="IdhhhBryzCa5eYqSPKAQfnVFpIg%3D", oauth_version="1.0", oauth_nonce="442087608", oauth_signature_method="HMAC-SHA1", oauth_consumer_key="5cb447d1fc4c3308e2c6531e45bcadf1", oauth_token="460633205c55d3f1806bcab04174ae09", oauth_timestamp="1334336020", Content-Length=0, Content-Range=bytes */*}
Verb: PUT
Complete URL: http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d
4639359 != 15125120
verifyUpload to http://174.129.125.96:8080/upload?ticket_id=5ea64d64547e38e5e3c121852b2d306d >>> Response
Success is not true!
Code: 308
Headers: {null=HTTP/1.1 308 Resume Incomplete, Range=bytes=0-4639359, Content-Length=0, Connection=close, Content-Type=text/plain, Server=Vimeo/1.0}
Body:
Then the code goes on to complete the upload and set video information (you can see that in my full code).
Edit 2: Tried removing the "%20" from the content-range and received this error making connection. I must use either "bytes%20" or not add "bytes" at all...
Exception in thread "main" org.scribe.exceptions.OAuthException: Problems while creating connection.
at org.scribe.model.Request.send(Request.java:70)
at org.scribe.model.OAuthRequest.send(OAuthRequest.java:12)
at autouploadermodel.VimeoTest.signAndSendToVimeo(VimeoTest.java:282)
at autouploadermodel.VimeoTest.sendVideoBytes(VimeoTest.java:130)
at autouploadermodel.VimeoTest.sendVideo(VimeoTest.java:105)
at autouploadermodel.VimeoTest.main(VimeoTest.java:62)
Caused by: java.io.IOException: Error writing to server
at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:622)
at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:634)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1317)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
at org.scribe.model.Response.<init>(Response.java:28)
at org.scribe.model.Request.doSend(Request.java:110)
at org.scribe.model.Request.send(Request.java:62)
... 5 more
Java Result: 1
Edit 1: Updated the code and output. Still need help!
I think your problem could simply be the result of this line:
request.addHeader("Content-Range", "bytes%20" + contentRange);
Try and replace "bytes%20" by simply "bytes "
In your output you see the corresponding header has incorrect content:
Headers: {
Content-Length=15125120,
Content-Type=video/mp4,
Content-Range=bytes%200-10485759/15125120 <-- INCORRECT
}
On the topic of Content-Range...
You're right that an example final block of content should have a range like 14680064-15125119/15125120. That's part of the HTTP 1.1 spec.
Here
String contentRange = Integer.toString(byteNumber + 1);
you start from 1 and not from 0 at the first iteration.
Here
request.addHeader("Content-Length", contentLength);
you put the entire file content length and not the length of the current chunk.
The vimeo API page says:
"The final step is to call vimeo.videos.upload.complete to queue up the video for transcoding. This call will return the video_id, which you can then use in other calls (to set the title, description, privacy, etc.). If you do not call this method, the video will not be processed."
I added this bit of code to the end and got it to work:
request = new OAuthRequest(Verb.PUT, "http://vimeo.com/api/rest/v2");
request.addQuerystringParameter("method", "vimeo.videos.upload.complete");
request.addQuerystringParameter("filename", video.getName());
request.addQuerystringParameter("ticket_id", ticket);
service.signRequest(token, request);
response = request.send();
Check this :
String contentRange="bytes "+lastBytesSend+"-"+ ((totalSize - lastBytesSend)-1)+"/"+totalSize ;
request.addHeader("Content-Range",contentRange);

DJ Native Swing javascript command problems

Using DJ Native Swing it is possible to show a web page within a java application. When you do this it is also possible to communicate from the browser to the java runtime environment using the "command" protocol. The documentation has a code snippet which demonstrates it's usage:
function sendCommand( command ){
var s = 'command://' + encodeURIComponent( command );
for( var i = 1; i < arguments.length; s+= '&' + encodeURIComponent( arguments[i++] ) );
window.location = s;
}
As it looks here it seems to be a regular GET request to an url using the command protocol instead of http. Although when I create and image, script tag or just and ajax get request there is no response and the breakpoint in the java runtime isn't triggered.
I don't want to set the window.location because I don't want to navigate away from the page I am currently at. Using the link to navigate to a command url does work though but it also navigates away from the current page. The page uses OpenLayers and dojo. (I have also tried dojo.io.script)
After some work I have found a neat way to communicate with the java runtime which doesn't trigger a refresh of the page every time there is communication. It is inspired on the way JSONP works to get around the cross domain restriction in most browsers these days. Because an iFrame will also trigger a command:// url it possible to do a JSONP like action using this technique. The code on the client side (browser):
dojo.provide( "nmpo.io.java" );
dojo.require( "dojo.io.script" );
nmpo.io.java = dojo.delegate( dojo.io.script, {
attach: function(/*String*/id, /*String*/url, /*Document?*/frameDocument){
// summary:
// creates a new tag pointing to the specified URL and
// adds it to the document.
// description:
// Attaches the script element to the DOM. Use this method if you
// just want to attach a script to the DOM and do not care when or
// if it loads.
var frame = dojo.create( "iframe", {
id: id,
frameborder: 0,
framespacing: 0
}, dojo.body( ) );
dojo.style( frame, { display: "none" } );
dojo.attr( frame, { src: url } );
return frame;
},
_makeScriptDeferred: function(/*Object*/args){
//summary:
// sets up a Deferred object for an IO request.
var dfd = dojo._ioSetArgs(args, this._deferredCancel, this._deferredOk, this._deferredError);
var ioArgs = dfd.ioArgs;
ioArgs.id = dojo._scopeName + "IoScript" + (this._counter++);
ioArgs.canDelete = false;
//Special setup for jsonp case
ioArgs.jsonp = args.callbackParamName || args.jsonp;
if(ioArgs.jsonp){
//Add the jsonp parameter.
ioArgs.query = ioArgs.query || "";
if(ioArgs.query.length > 0){
ioArgs.query += "&";
}
ioArgs.query += ioArgs.jsonp
+ "="
+ (args.frameDoc ? "parent." : "")
+ "nmpo.io.java.jsonp_" + ioArgs.id + "._jsonpCallback";
ioArgs.frameDoc = args.frameDoc;
//Setup the Deferred to have the jsonp callback.
ioArgs.canDelete = true;
dfd._jsonpCallback = this._jsonpCallback;
this["jsonp_" + ioArgs.id] = dfd;
}
return dfd; // dojo.Deferred
}
});
When a request is sent to the java runtime a callback argument will be supplied and a webBrowser.executeJavascript( callbackName + "(" + json + ");" ); action can be executed to trigger the callback in the browser.
Usage example client:
dojo.require( "nmpo.io.java" );
nmpo.io.java.get({
// For some reason the first paramater (the one after the '?') is never in the
// paramater array in the java runtime. As a work around we stick in a dummy.
url: "command://sum?_",
callbackParamName: "callback",
content: {
numbers: [ 1, 2, 3, 4, 5 ].join( "," )
},
load: function( result ){
console.log( "A result was returned, the sum was [ " + result.result + " ]" );
}
});
Usage example java:
webBrowser.addWebBrowserListener(new WebBrowserAdapter() {
#Override
public void commandReceived(WebBrowserCommandEvent e) {
// Check if you have the right command here, left out for the example
// Parse the paramaters into a Hashtable or something, also left out for the example
int sum = 0;
for( String number : arguments.get( "numbers" ).split( "," ) ){
sum += Integer.parseInt( number );
}
// Execute the javascript callback like would happen with a regular JSONP call.
webBrowser.executeJavascript( arguments.get( "callback" ) + "({ result: " + sum + " });" );
}
});
Also with IE in the frame I can highly recommend using firebug lite, the dev tools for IE are not available.

Categories