I am trying to update header data of some Rackspace objects i have uploaded. Example header attribute like: X-Object-Meta-name
But to do that, currently i need to download the whole object and parse the header from the downloaded object. Then do some checking and update if necessary and then upload the object again. But this makes the updating process very slow.
Is there a way to only download the header part of an object and updating it alone? Thanks in advance!
https://github.com/jclouds/jclouds/blob/master/apis/openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/features/ObjectApi.java#L207
If you give it a map with "name"->"the updated header value", it should update the header and automatically add the x-object-meta- prefix.
Is there a way to only download the header part of an object and updating it alone?
I am not a Java developer, but the Cloud Files API is a RESTful one, so I will provide examples using curl. If you are using a library then you may want to edit your question to include which library as many of them abstract these opperations and a better answer can probably be provided in the context of that library.
To download the headers without the object content, perform a HTTP HEAD request.
$ curl -I -XHEAD -H'X-Auth-Token:******' \
> https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_******/container/object
HTTP/1.1 200 OK
Content-Length: 400
Accept-Ranges: bytes
Last-Modified: Tue, 21 Apr 2015 12:06:23 GMT
Etag: 81dc9bdb52d04dc20036dbd8313ed055
X-Timestamp: 1429617982.70468
X-Object-Meta-Foo: Bar
Content-Type: text/html
X-Trans-Id: txd337e4634c98475baf1a4-0055363d42dfw1
Date: Tue, 21 Apr 2015 12:06:26 GMT
To update just the headers on the object, you can then perform a HTTP POST request.
$ curl -i -XPOST -H'X-Auth-Token:******' \
> -H'X-Object-Meta-Foo: Bar' \
> -H'X-Object-Meta-Foo2: Bar2' \
> https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_******/container/object
HTTP/1.1 202 Accepted
Content-Length: 76
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txc262dfe86727440cbfcb1-0055363d5cdfw1
Date: Tue, 21 Apr 2015 12:06:53 GMT
<html><h1>Accepted</h1><p>The request is accepted for processing.</p></html>
Performing another HEAD requeset will show that both the headers are now present.
$ curl -I -XHEAD -H'X-Auth-Token:******' \
> https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_******/container/object
HTTP/1.1 200 OK
Content-Length: 400
Accept-Ranges: bytes
Last-Modified: Tue, 21 Apr 2015 12:06:53 GMT
Etag: 81dc9bdb52d04dc20036dbd8313ed055
X-Timestamp: 1429618012.98354
X-Object-Meta-Foo: Bar
X-Object-Meta-Foo2: Bar2
Content-Type: text/html
X-Trans-Id: txdd9365b54e8f4d8c8451d-0055363d6adfw1
Date: Tue, 21 Apr 2015 12:07:06 GMT
Related
Two part question that may or may not be related to each other.
I am running Stackify Prefix v3.0.28 for a Java application on Win10 and it generally seems to work OK: I can see the traces of various actions in our application.
Part 1:
When navigating to any page of our application I get two failed requests to load JS files:
http://127.0.0.1:2/scripts/sfclient.xhr.min.js
http://127.0.0.1:2/scripts/sfclient.perf.prefix.min.js
Both of these requests fail with ERR_CONNECTION_REFUSED. Those script references are not in my JSP page so I assume they are injected by Prefix.
Here is the raw HTML that tries to load the 2 scripts:
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><script src="http://127.0.0.1:2/scripts/sfclient.xhr.min.js"></script>
<script>var SPerfLib = window.SPerfLib || {}; SPerfLib.RequestId = '54fd58d1-7f7e-d3a4-0001-331676a83598'; if(!SPerfLib.isAttached) { document.addEventListener('DOMContentLoaded', function() { var l = document.createElement('script'); l.src = 'http://127.0.0.1:2/scripts/sfclient.perf.prefix.min.js'; document.body.appendChild(l);}); SPerfLib.isAttached = true;}</script>
I have tried looking for configuration options, but found none. I was not sure if the scripts should be server from port 2 or not. The Prefix trace output is from port 2012 and that seems correct.
I tried uninstalling and re-installing Prefix, but with the same results. There does not seem to be any later version of Prefix to try.
How do I get those scripts to load successfully?
Part 2:
On one particular page we have an XHR to retrieve some JSON data. The server is returning data correctly, but it is somehow deleted before it arrives at the browser. The response headers show status 200 but 0 bytes content-length, which then causes some of our JS on the page to fail. If I run the same thing w/o Prefix everything works as expected - status is still 200, but content-length is 37 and JSON payload is visible.
This is the response header for the XHR when Prefix is in play (note content-length: 0)
cache-control: no-cache, must-revalidate
content-language: en-US
content-length: 0
content-type: text/html
date: Mon, 31 Aug 2020 14:19:24 GMT
expires: Thu, 01 Jan 1970 00:00:00 GMT
last-modified: Mon, 31 Aug 2020 14:19:24 GMT
pragma: no-cache
server: WildFly/10
status: 200
x-powered-by: Undertow/1
x-powered-by: JSP/2.3
x-stackifyid: V1|8bbdce1c-a507-bbdc-0001-3378bff33740|
If I remove the Stackify agent from the JVM options and disable the profiler, then the response header looks like this:
cache-control: no-cache, must-revalidate
content-language: en-US
content-length: 37
content-type: text/html;charset=UTF-8
date: Mon, 31 Aug 2020 14:25:12 GMT
expires: Thu, 01 Jan 1970 00:00:00 GMT
last-modified: Mon, 31 Aug 2020 14:25:12 GMT
pragma: no-cache
server: WildFly/10
status: 200
x-powered-by: Undertow/1
I'm appreciative of any suggestions!
These issues you are having with Prefix are known issues with Prefix. We are working on a complete re-write of Prefix (a reason why there has been such a big delay since our last release) and these items are things we are getting fixed in the new Prefix version. We are getting very close to releasing a Beta for Prefix, if you would like to be on the list to give the Prefix beta a try email the Stackify Support Team support#stackify.com
For some reason the previously working code stopped working and server started to respond with 416.
Here are the logs of HTTP client during failing interaction:
-------------- REQUEST --------------
GET https://www.googleapis.com/drive/v3/files/0B02Nopv3SQOvOVNKaDIwTEZ3MHd?alt=media
Accept-Encoding: gzip
Authorization: <Not Logged>
Range: bytes=0-33554431
User-Agent: My app Google-API-Java-Client Google-HTTP-Java-Client/1.22.0 (gzip)
-------------- RESPONSE --------------
HTTP/1.1 416 Requested range not satisfiable
Alt-Svc: quic=":443"; ma=2592000; v="39,38,37,35"
Server: UploadServer
Cache-Control: private, max-age=0
Content-Range: bytes */0
X-GUploader-UploadID: AEnB2UqBx9B09Lnr8tG761gdoz3DkhHSNO_OzHh1LkU6B2908v17rnBGQZSNW4ZVTjbRdFtvPWWIqZGdtSrTo6ZWN7YW9nxf6d
Vary: X-Origin
Vary: Origin
Expires: Mon, 11 Sep 2017 15:23:20 GMT
Content-Length: 225
Date: Mon, 11 Sep 2017 15:23:20 GMT
Content-Type: application/json; charset=UTF-8
I was trying to download a file which is around 200000 bytes, so I thought meaning of "chuck size" changed somewhere, so it could not give 33554431 bytes of a 282177 byte file. Tried changing that to a smaller value, but no success.
Drive.Files.Get get = drive.files().get(file.getId())
MediaHttpDownloader downloader = get.getMediaHttpDownloader()
downloader.directDownloadEnabled = false
localFile.newOutputStream()
get.executeMediaAndDownloadTo(stream)
Direct download does not work either, it just downloads "0" bytes.
Does anyone know how to overcome this issue?
416 Range Not
Satisfiable
error means the server is not able to serve the requested ranges. The
most likely reason is that the document doesn't contain such ranges,
or that the Range header value, though syntactically correct, doesn't
make sense.
One of the resolutions that may provide from this forum is to:
Add "Accept-Ranges: none" to our response headers.
It appeared to be a web interface when using Firefox. It uploaded "empty" files in certain cases.
https://productforums.google.com/forum/#!topic/drive/S03wEknc75g;context-place=forum/drive
I am a new programmer i am trying to build an app with Json.
If i use this URL doesn't work . http://zsuzsafodraszat.hostzi.com/boltok.json
if i Use this, my app working. https://api.myjson.com/bins/3zm8i
Both Json files exactly the same.
Can you help me what i am doing wrong ? Maybe bad extension or web000 is not a good service for Json ? Can you give me some good free json hosting ? Thanks
Those 2 urls do not have the same content or the same headers. You can see this if run curl commands from the command line:
$ curl -i "http://zsuzsafodraszat.hostzi.com/boltok.json"
HTTP/1.1 200 OK
Date: Wed, 13 Apr 2016 22:52:50 GMT
Server: Apache
Last-Modified: Wed, 13 Apr 2016 16:48:23 GMT
Accept-Ranges: bytes
Content-Length: 1020
Connection: close
Content-Type: application/json
??{"Aldi":"http://catalog.aldi.com/emag/hu_HU/print/Online_katalogus_04_07/Online_katalogus_04_07.pdf",
"Lidl":"http://www.lidl.hu/statics/lidl-hu/ds_doc/HU_HHZ_kw14_2016.pdf",
"Spar":"http://ajanlatok.spar.hu/view/download/?d=1279",
"Penny":"https://view.publitas.com/16538/136265/pdfs/016f82fb5b00bc97b5a8c35f512d89b01cd3e3ce.pdf",
"Coop":"https://view.publitas.com/2556/133497/pdfs/16603d7e9bf30e8a8a4efec7f01d3fa2caf92fe0.pdf",
"Auchan":"http://www.lidl.hu/statics/lidl-hu/ds_doc/HU_HHZ_kw14_2016.pdf"}
$ curl -i "https://api.myjson.com/bins/3zm8i"
HTTP/1.1 200 OK
Server: nginx/1.5.8
Date: Wed, 13 Apr 2016 22:52:56 GMT
Content-Type: application/json
Content-Length: 500
Connection: keep-alive
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
{"Aldi":"http://catalog.aldi.com/emag/hu_HU/print/Online_katalogus_04_07/Online_katalogus_04_07.pdf","Lidl":"http://www.lidl.hu/statics/lidl-hu/ds_doc/HU_HHZ_kw14_2016.pdf","Spar":"http://ajanlatok.spar.hu/view/download/?id=1279","Penny":"https://view.publitas.com/16538/136265/pdfs/016f82fb5b00bc97b5a8c35f512d89b01cd3e3ce.pdf","Coop":"https://view.publitas.com/2556/133497/pdfs/16603d7e9bf30e8a8a4efec7f01d3fa2caf92fe0.pdf","Auchan":"http://www.lidl.hu/statics/lidl-hu/ds_doc/HU_HHZ_kw14_2016.pdf"}
As you can see, one of them has a couple of junk bytes at the beginning that my terminal is displaying as question marks. Also the http headers are different. The Content-Lengths are wildly different too. Did you use something other than a plain text editor to create the json payload in the failing example?
Try removing the junk characters and adding these http headers:
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
I am trying to download an image from URL. I use the Apache Commons library:
org.apache.commons.io.FileUtils
.copyURLToFile(
new URL(
"https://lh3.ggpht.com/AXYMUV5cpne2vE9U3X8x87HgrbwijwiG50_yOnehU2MUPKDoJky-BKFOPZzZ07Pug8U=h230"),
new File("test.png"));
While opening the image it says not an PNG file. Is this a good way to download images from url to local?
Updated .
Thankyou guys for your answers i undestand that i need to get the connection.getContentType(); and then save it as jpg or png as it is.
if you check the url, It is JPEG file. Try to execute program after renaming file name to test.jpeg..
Checking for the content type see this SO Question.
Then use this to save the file to the right type.
Running wget on this URL and looking at the HTTP response, looks like the file type is explicitly specified as JPEG:
--2012-06-27 16:55:30-- https://lh3.ggpht.com/AXYMUV5cpne2vE9U3X8x87HgrbwijwiG50_yOnehU2MUPKDoJky-BKFOPZzZ07Pug8U=h230
Resolving lh3.ggpht.com... 74.125.237.108, 74.125.237.106, 74.125.237.107, ...
Connecting to lh3.ggpht.com|74.125.237.108|:443... connected.
HTTP request sent, awaiting response...
HTTP/1.0 200 OK
Access-Control-Allow-Origin: *
ETag: "v1"
Expires: Thu, 28 Jun 2012 06:50:40 GMT
Content-Disposition: inline;filename="unnamed.jpg"
Content-Type: image/jpeg
X-Content-Type-Options: nosniff
Date: Wed, 27 Jun 2012 06:50:40 GMT
Server: fife
Content-Length: 46597
X-XSS-Protection: 1; mode=block
Cache-Control: public, max-age=86400, no-transform
Age: 290
Connection: Keep-Alive
Length: 46597 (46K) [image/jpeg]
when I get the following url with curl
curl -D headers.http "http://www.springerlink.com/index/10.1007/s00453-007-9157-8"
the file headers.http contains a "Location" header:
HTTP/1.1 302 Found
Date: Tue, 27 Oct 2009 17:00:20 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: http://www.springerlink.com/link.asp?id=c104731297q64224
Set-Cookie: CookiesSupported=True; expires=Wed, 27-Oct-2010 17:00:20 GMT; path=/
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 173
but when I used the apache httpclient library this "Location:" header was missing (?).
int status = httpClient.executeMethod(method);
if(status!=HttpStatus.SC_OK &&
status!=HttpStatus.SC_MOVED_TEMPORARILY &&
status!=HttpStatus.SC_MOVED_PERMANENTLY
)
{
throw new IOException("connection failure for "+url+" status:"+status);
}
Header header=method.getResponseHeader("Location");
if(header==null )
{
for(Header h:method.getResponseHeaders())
{
LOG.info(h.toString());
}
throw new IOException(
"Expected a redirect for "+url
);
}
I've listed the headers below:
INFO: Date: Tue, 27 Oct 2009 17:05:13 GMT
INFO: Server: Microsoft-IIS/6.0
INFO: X-Powered-By: ASP.NET
INFO: X-AspNet-Version: 2.0.50727
INFO: Set-Cookie: ASP.NET_SessionId=js1o5wqnuhuh24islnvkyr45; path=/; HttpOnly
INFO: Cache-Control: private
INFO: Content-Type: text/html; charset=utf-8
INFO: Content-Length: 17245
uhh ???
What's going on is that with curl , you are getting a 302 which is actually a redirect, to the URL in the location header.
With the Apache httpclient it is doing the redirect for you, and returning the headers from the request to the redirected-to location.
To demonstrate this try
curl -D headers.http "http://www.springerlink.com/link.asp?id=c104731297q64224"
and compare the response.
edit: There are actually about 4 redirects in there if you follow each location header through with curl.
http://www.springerlink.com/index/10.1007/s00453-007-9157-8 is actually a redirect. Since the -D option means "headers only", the first one is not redirecting to the specified Location: ..., while the second one is. Take a look at the Content-Length, it's much different.
What happens when you leave out the -D?
Add this
method.setFollowRedirects(false);
Before you execute the method.
HttpClient follows the redirect automatically by default but Curl doesn't.