We Keep Coding

How to download csv from uk government coronavirus api

I've been trying to download the csv from the UK government coronavirus API. you can download the file manually by using the URL: https://api.coronavirus.data.gov.uk/v1/data?filters=areaType=overview&structure=%7B%22areaName%22:%22areaName%22,%22date%22:%22date%22,%22newCasesBySpecimenDate%22:%22newCasesBySpecimenDate%22,%22cumCasesBySpecimenDate%22:%22cumCasesBySpecimenDate%22%7D&format=csv
I've tried multiple methods of downloading files, such as JavaIO:
InputStream in = null;
try {
in = new URL("https://api.coronavirus.data.gov.uk/v1/data?filters=areaType=overview&structure=%7B%22areaName%22:%22areaName%22,%22date%22:%22date%22,%22newCasesBySpecimenDate%22:%22newCasesBySpecimenDate%22,%22cumCasesBySpecimenDate%22:%22cumCasesBySpecimenDate%22%7D&format=csv").openStream();
} catch (IOException e) {
e.printStackTrace();
}
try {
Files.copy(in, Paths.get("test.csv"), StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
e.printStackTrace();
}
and Java NIO:
ReadableByteChannel readableByteChannel = Channels.newChannel(url.openStream());
FileOutputStream fileOutputStream = null;
try {
fileOutputStream = new FileOutputStream("test.txt");
} catch (FileNotFoundException e) {
e.printStackTrace();
}
FileChannel fileChannel = fileOutputStream.getChannel();
try {
fileOutputStream.getChannel()
.transferFrom(readableByteChannel, 0, Long.MAX_VALUE);
} catch (IOException e) {
e.printStackTrace();
}
however, it only shows this in the file:
‹ 0w®_ÿ}ZKŽX5Üs
àHm÷ÏÞ;$6ˆŒÈe‘€HâöTy²L=)Š”y»ÝŸru{^þ~}ùååãëxÿòåu|zý÷ǗϯŸøï׿^ÿðñõÓOüñïÿ|üÆ¿ûíÓ‡/¯ï¿ÿùç?Þÿùq,[önNü3,Ç\Çæ ³1ÛN·wï£pvÆjÛ\Ï:Kâö˜§66_ez=ì¸}½Vœµ%®Æò•°rY•™Ä%p½`–õÔ¸ ®xŽ™§VKœ•8ÏÒ¶†Ï¨nZ˜ö<’1¦ÁË^ßÆÙ;®bÃ;V>•Ÿ³±V]\T[*ÜBÜÜž¶Âæ–¸M¿,œ×Ìjk\8ë8݈„!lE÷\'µuI¯¸³*¦KÜV+±¼²M³Ýc#`Ka1¥¼ÆnØ6%l""|ì8=õˆô™c¯X-ã5óiïÑgŸl CA2x4¼RGÂ÷èêÑyæ–Q˜HK„ °ˆ˜z5Dar¤WõÔG–ö‰X[ÃP[{ƒQE©i"Õ¦UXTF4†Êߣ ¬OJ‚H¬Y¾gËMÁbÈ7d²©tö’ÄÎ;!mûÊas u¤{Q°ìÁL°
Q#Éì‘YXTÂœ«9JP–Œ]†Ë¹qú¤ˆ‚WŸ‘`—1%{ùìBç·7µCRBÓ`Žo'‘æ#­šËÕ#I¼0À®-ˆ($Úe‘µED‰ªQ#ÊoJT¢à3Týt½* PZáHomjØòÑz°9¨æXÈò–(¤X¢¢EL¢9ÁµìÌ#½
"Š]ð½ùnix(Nùðõ…Íʉ‚Òž¸®B;hYQ9 TZʃ»5ãáŒ÷" ª·½#îjÜß ÑÔgœØ•î
µ¢w4X³ö*èçÞÅ^å-íû ëk6Ûµî¶!­§|ŠèëZm=ÅËÖÓCç=M2çZöPC ‡†½3$
¾?Œ#Œ{#á
¦ös”H‰¼ßW2¥1L¸Í5
Ü4™Ñ¾MÈR HMk1LµÅmCÔ¦!*k¶DázÈ»£q…Û~
NJ…<ãß
‘ATÐzäÄ:ûa-§Vh—¹-€
 îÖÞ‚ç'³~ÕI½<ôÃÊNéÓIÏb
7¹õ»è7Ó;ÎÛÎ Ÿ¬“^ʃZ£¼Î-Õ=¡Ù–‘¦2šè Øýèýx%³Ê -VHÔ¢à!
ÂMzž²È!) <wëµèù¾Jê<+ñ:Õg!ý R‚Nˆá‘ 'Xÿ³]‡r膙¬E5ÄëÓÑ›š6ýÔ$j£i’(g—7X׃ñkP/¹UêHƒfò®TR¸4 b Å“M>:,U‹=K®Rrž—§Dmô(p¨!çS‚Ð1aðO!ñ BIð6jG ‚\Üñi ¢HâJf­
‡F:‡7*Aë+Â#o7G#§»Ò-A66/}ôKžŽú(¸RÍ)#|å:á“¥ø¥ÉBÙ4µ§W*ÈÄ (R$ÔMÑz»#…AsJ€ƒØn¯³TƒLÒ²x:t5ÚO‘ ùÃJ¤qqPŸ2vàŸžÜÎÜdf‚š34[Úþ)j†½¥0oòOS²eu‚~íÂÚ;K‚‚ú˜ ü#‚œ}#UKoÇV“Û•…t&NVµ(r$è·AEZ©É•Ð)­4>;Ð÷KrŠ¢7Ô­]·û‚#HŠªzk¾¾{A¬J{/úi–¢ßº¬ÒŒÒ†CßlºÀ¦’]uõ
ô'šc7éÌEãt½ãátðø†MJ%A`•ƒž}‘žävWÜ0.»GºœâÆ&MO‰âD’µÐhͤ«8ù1æ'aÚ®€¤æÑn˜ô:mÇå®Aœ/2_X1S¢&OˆµêÌÔ(
úƒŒ©=]ÚΉ4P…&U¢èù»V¶bŽ"½4çx¥jÙ$ôr)=몛É9dù,iª¶?«–…Œ!Õ“‰zJ£àù8Ø×̃·øhàì[ͽ,oÃe›sOÔ~Iãs‡¨PLE…“œg¸˜b…ø w jÉv$oõ9G¸\¢PÅë-ô˜$oÃU´ýéƒõqÇg#åy°ÞyF¢"}K‚cÌÔ ü’(:´‘_`¤ÔÖßjåSÇví¯+œyË¡-i=1É]èòT{—•¶ó±Æ|VZFMšW,Ÿd%zócšôÇ&|Áü­íò·ñÍZ¦fýD-N°ÖìR•–w””X3Vh]ã&: ‰â{ÿÞèé%jßVÿF‡&ãx§SÌBCO¥QèÞØÒO¨íX…¾¥6gó™Sæ=•óí èÒ_v9gé;´ïù‚†•Ñ2sŒÓâ‹B¼EN9'NÝWT×zs ¡#b½Ò[¢6wäs\ËÛ8È9A9‰Xºª¡ ç8ñ™]J9Ç;85
YAÎAïaœy…ºÑ‚œ“¸<AçðªF¡ç
ˆ—Hm}_œ8ÄvíUø~ƒ¿&#¨ârΦ]¨ðEßóê^ W¹#ÇÏ'ö½¤^=-Ä>hÛd'ß6’ívÅ”ùÎáÓ å‰ë8pßð¥Ð"ícÇ8ýú„dðå>’ƃ„s‚óMFÚ8öO¹:µ#79‘Ñ]¥ã¶¹Ï¢=S»Óê><*Ó§#×#Ž õé-ê ܇h¢æѧƒÇûmŸÚO äzá/Ü4ßV‚HREï#¨!“Cmç—‹p óbQÛù}žgÑ,ùJ[5^ü%ËÏID(a
9ßñËCF•n{éíòês’èÃvÍç{s½’ó*ä0¶R:“¿s€ìæ kIg¾Mš7
Aüj1÷õÙÛ)À)3ú!´uj²á$ŸsÞ® apPz'4½ÕÒ‡“ï‘wâ(ÏÄ÷.ª„šÒdŠñ¥(ÐÉ5±o#5„/‚÷·…ô‰Ø‰€Aøm¶ç^òÈdÞ˜z‰ÅçPéöWdE½˜œ}¯<…­„. Žlx£Ë!“ð7jdºÞ‡«¡Ûº
^ŸùkQCåÖº³\¤üŽÃþò34Ô·ne5ú\·™ª?Y¬fzF~žüßú³áK¯¡xŸ¾îǯýøµ¿æã×xüê_×ãW:K¥¯dYŠ_¡¯†L?£¯dX=~Ej=|¥¯dbš?~]_§þ:ya?}Å7|ý:7é…)
I then moved on to trying to make an http request, which I used this code:
URL url = new URL("https://api.coronavirus.data.gov.uk/v1/data?filters=areaType=overview&structure=%7B%22areaName%22:%22areaName%22,%22date%22:%22date%22,%22newCasesBySpecimenDate%22:%22newCasesBySpecimenDate%22,%22cumCasesBySpecimenDate%22:%22cumCasesBySpecimenDate%22%7D&format=csv");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
int status = con.getResponseCode();
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer content = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
content.append(inputLine);
}
in.close();
con.disconnect();
which only returns
0w�_�}ZK�X5�s�Hm���;$6��e��H��Ty�L=)��y�ݟru{^�~}�����x���u|z��Ǘϯ���׿^�����O����|�Ə���Ӈ/����ç?���q,[�nN�3,�\�� �1�N�w�pv�j�\�:K����66_ez=�}�V��%���rY���%p�`��Ը �x���VK���8����ϨnZ��<�1���^���;�b�;V>�����V]\T[*�B�܁���斸M�,���jk\�8�8��!lE��\'�uI����*�K�V+���M���c#`Ka1���n�6%l""|�8=����c�X-�5�i��g�l CA2x4�RG�����y�Q�HK� ���z5Dar�W�����X[�P[{�Q�E�i"��UXTF�4��ߣ �OJ�H�Y�g�M�b�7d��t����;!m��asu�{�Q���L�Q#��YXT���9JP��]����q����W��`�1%{��B�7�CRB�`�o'���#����#I�0��-��($ځe��ED��Q�#�oJT��3T�t�*�PZ�Homj���z�9��X��(�X��EL�9����#�"�]��nix(N������ʉ�Ґ���B;hY�Q9 TZ���5���" ���#�j�� ��g������w4X����*����^�-�� �k6۵�!��|���Zm=����C�=M2�Z�PC ���3$�?�#�{#���s�H���W2�1L��5�4�ѾM�R�HMk1L��mCԦ!*k�D�zȻ�q��~NJ�<��AT�z��:�a-�Vh��-����ނ�'�~�I�<���N��I�b7����7�;��� ���^ʃZ����-�=�ٖ��2�� ����x%�ʠ-VHԢ�!�Mz���!) <w����J�<+�:Õg!� R�N�ᑠ'X��]�r����E5���ћ�6���$j�i�(g�7X׃�kP/�U�H�f�TR�4 b�œM>:,U�=K��Rr����Dm�(p�!�S��1a�O!� BI�6jG �\��i �H�Jf��F:�7*A��#o7G#���-A66/}�K���(�R�)#|�:ᓥ���B�4��W*�� (R$�MÑz�#�AsJ���n��T�LҲx:t5�O���� ��J��qqP�2v������df��34[��)j���0o�OS�eu�~��ڍ;K���� �#��}#UKo�V�ە�t&NV�(r$�AEZ�ɕ�)�4>;��Kr��7ԭ]���#H��zk���{A�J{/�i��ߺ�Ҍ��҆C�l����]u��'�c7��E��t���t���MJ%A`���}���vW�0.�G����&M��D���hͤ�8�1�'aڮ����n��:m��A�/2_X1S�&O�����(����=]�Ή4P�&U����V�b�"�4�x�j�$�r)=몛�9d�,i��?����!Ս��zJ���8��̃��h��[ͽ,o�e�sO�~I�s��PLE���g��b���w j�v$oõ9G�\�P��-��$o�U����q�g#�y��yF�"}K�c�Ԡ��(:��_`����j�S�v�+�yˡ�-i=1�]��T{������|VZFM�W,�d%z�c����&|�������Z�f�D-N���R��w��X3Vh]�&: ��{����%j�V�F�&�x�S�BCO�Q����O��X���6g�S�=���� ��_v9g�;�������2s���B�EN9'N�W�T�z�s �#b��[�6w�s\��8�9A9�X��� �8�]J9�;85YA�A�a�y��т���<A��F����Hm}_�8�v�U�~��&#���rΦ]��E���^�W�#��'����^=-�>h�d'�6��vŔ���� ��8p����"�c�8���d��>�ƃ�s���MF�8��:�#79��]����Ϣ=S���><�*ӧ#�#����-�� ܇h��ѧ���m��O �z�/�4�V�HRE�#�!�Cm痋p �bQ��}�g�,�J[5^�%��ID(a9���CF�n{����s���v��{s���*�0�R:��s���kIg�M�7A�j1����)�)3�!�uj��$�s a�pPz'4��҇��w�(���.����d��(��5�o#5�/�����؉�A�m��^��dޘz���P��WdE���}�<���. �lx��!��7jd�އ��ۺ^���kQC�ֺ�\�������34��ne5�\���?Y�fzF~������K��x���ǯ�������x��_��W:K��dY�_���L?��dX=~Ej=|��db�?~]�_��:ya?}�7|�:7��)
when outputting content.
Can anyone help me?

In violation of the HTTP protocol, the server returns the csv data with Content-Encoding: gzip, even if you explicitly send an Accept-Encoding: identity header. You will need to decompress the response body.
This constraint is actually documented in the Developers guide for this API:
All API responses are compressed using GZip. The request client must therefore accept GZip encoded content.
Something like this should work, and continue to work if the server starts sending uncompressed responses against our (now explicit) request:
URLConnection connection;
try {
connection = new URL("...").openConnection();
connection.setRequestProperty("Accept-Encoding", "gzip");
connection.connect();
} catch (IOException e) {
e.printStackTrace();
}
InputStream in = connection.getInputStream();
String encoding = connection.getContentEncoding();
if ("identity".equals(encoding)) {
// Nothing to do.
} else if ("gzip".equals(encoding)) {
in = new GZIPInputStream(in);
} else {
throw IOException("Unsupported encoding " + encoding);
}
try {
Files.copy(in, Paths.get("test.csv"), StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
e.printStackTrace();
}
Note that neither this code, nor your own, closes the connections and streams properly. I recommend to use try-with-resources to close things gracefully even in case of an exception.
You could save yourself some headaches by using a third-party HTTP library, such as OkHttp or Apache HttpClient.

How to read string from php to android studio (java)

I don't know why i get a null value when i call the GetPHPData() function. The "out" variable returns nothing (""). I make a Toast.makeTest and it returns empty string. Please help. This is my code:
public class PHPConnect extends Activity
{
String url = "http://122.2.8.226/MITBookstore/sqlconnect.php";
HttpURLConnection urlConnection = null;
String out = null;
public String GetPHPData()
{
try {
urlConnection = (HttpURLConnection) new URL(url).openConnection();
urlConnection.setConnectTimeout(5000);
urlConnection.setReadTimeout(10000);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
try
{
BufferedReader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
out = readStream(in);
}
catch (Exception e)
{
Toast.makeText(getApplicationContext(),e.getMessage(),Toast.LENGTH_SHORT).show();
}
finally
{
urlConnection.disconnect();
return out;
}
}
private String readStream(BufferedReader is)
{
try
{
ByteArrayOutputStream bo = new ByteArrayOutputStream();
int i = is.read();
while(i != -1)
{
bo.write(i);
i = is.read();
}
return bo.toString();
} catch (IOException e)
{
return e.getMessage();
}
}
}
By the way, im running a wamp server and I port forwarded my router, on local host, the url works, but on remote connection, it won't return a string. You can try out the url, the result is: "This is the output:emil"

Can you please try below piece of code which is working for me, also add INTERNET permission in android manifest file. Still if it is not working then may be issue with server end then try to debug it.
URL url;
try {
url = new URL("myurl");
HttpURLConnection urlConnection = (HttpURLConnection) url
.openConnection();
InputStream in = urlConnection.getInputStream();
InputStreamReader isw = new InputStreamReader(in);
int data = isw.read();
while (data != -1) {
char current = (char) data;
data = isw.read();
System.out.print(current);
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

Nothing is displayed even after using try catch block

this block neither gives an error in the log nor does it put any value in the string body
try {
int roll = 00111502713;
//url defined is completely fine//
URL url = new URL("someurl");
URLConnection con = url.openConnection();
InputStream in = con.getInputStream();
encoding = con.getContentEncoding();
encoding = encoding == null ? "UTF-8" : encoding;
try {
body = IOUtils.toString(in, encoding);
Toast.makeText(FirstActivity.this, "Body found", Toast.LENGTH_SHORT).show();
} catch (Exception e) {
Log.e("Log_tag", "Hahaha");
}
} catch (Exception ex) {
// System.out.println("This shouldn't have happened");
Log.e("Log_Tag", "Really?");
}

You have to call:
Toast.makeText(FirstActivity.this,"Body found",Toast.LENGTH_SHORT).show();

FileNotFoundException on HttpsURLConnection with POST and unmutable doOutput variable

I'm trying to POST some data to an https url, in my android application, in order to get a json format response.
I'm facing two problems:
is = conn.getInputStream();
throws
java.io.FileNotFoundException
I don't get if i do something wrong with HttpsURLConnection.
The second problem arose when i debug the code (used eclipse); I set a breakpoint after
conn.setDoOutput(true);
and, when inspecting conn values, I see that the variable doOutput remain set to false and type GET.
My method for https POST is the following, where POSTData is a class extending ArrayList<NameValuePair>
private static String httpsPOST(String urlString, POSTData postData, List<HttpCookie> cookies) {
String result = null;
HttpsURLConnection conn = null;
OutputStream os = null;
InputStream is = null;
try {
URL url = new URL(urlString);
conn = (HttpsURLConnection) url.openConnection();
conn.setReadTimeout(10000);
conn.setConnectTimeout(15000);
conn.setRequestMethod("POST");
conn.setUseCaches (false);
conn.setDoInput(true);
conn.setDoOutput(true);
if(cookies != null)
conn.setRequestProperty("Cookie",
TextUtils.join(";", cookies));
os = conn.getOutputStream();
BufferedWriter writer = new BufferedWriter(
new OutputStreamWriter(os, "UTF-8"));
writer.write(postData.getPostData());
writer.flush();
writer.close();
is = conn.getInputStream();
BufferedReader r = new BufferedReader(
new InputStreamReader(is));
StringBuilder total = new StringBuilder();
String line;
while ((line = r.readLine()) != null) {
total.append(line);
}
result = total.toString();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (ProtocolException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (os != null) {
try {
os.close();
} catch (IOException e) {
}
}
if (is != null) {
try {
is.close();
} catch (IOException e) {
}
}
if (conn != null) {
conn.disconnect();
}
}
return result;
}
A little update: apparently eclipse debug lied to me, running and debugging on netbeans shows a POST connection. Error seems to be related to parameters i'm passing to the url.

FileNotFoundException means that the URL you posted to doesn't exist, or couldn't be mapped to a servlet. It is the result of an HTTP 404 status code.
Don't worry about what you see in the debugger if it doesn't agree with how the program behaves. If doOutput really wasn't enabled, you would get an exception obtaining the output stream.

Java, FileNotfound Exception, While reading conn.getInputStream()

Please tell me some one, How to resolve this problem,
Sometime I am getting Filenotfound Exception and Some time this code working fine.
Below is my code,
public String sendSMS(String data, String url1) {
URL url;
String status = "Somthing wrong ";
try {
url = new URL(url1);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
conn.setRequestProperty("User-Agent","Mozilla/5.0 ( compatible ) ");
conn.setRequestProperty("Accept","*/*");
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
// Get the response
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String s;
while ((s = rd.readLine()) != null) {
status = s;
}
rd.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
wr.close();
} catch (MalformedURLException e) {
status = "MalformedURLException Exception in sendSMS";
e.printStackTrace();
} catch (IOException e) {
status = "IO Exception in sendSMS";
e.printStackTrace();
}
return status;
}

Rewrite like this and let me know how you go... (note closing of reading and writing streams, also the cleanup of streams if an exception is thrown).
public String sendSMS(String data, String url1) {
URL url;
OutputStreamWriter wr = null;
BufferedReader rd = null;
String status = "Somthing wrong ";
try {
url = new URL(url1);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
conn.setRequestProperty("User-Agent","Mozilla/5.0 ( compatible ) ");
conn.setRequestProperty("Accept","*/*");
wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data);
wr.flush();
wr.close();
rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String s;
while ((s = rd.readLine()) != null) {
status = s;
}
rd.close();
} catch (Exception e) {
if (wr != null) try { wr.close(); } catch (Exception x) {/*cleanup*/}
if (rd != null) try { rd.close(); } catch (Exception x) {/*cleanup*/}
e.printStackTrace();
}
return status;
}

This issue seems to be known, but for different reasons so its not clear why this happend.
Some threads would recommend closing the OutputStreamWriter as flushing it is not enough, therefor i would try to clos it directly after fushing as you are not using it in the code between the flush and close.
Other threads show that using a different connections like HttpURLConnection are avoiding this problem from occuring (Take a look here)
Another article suggests to use the URLEncoder class’ static method encode. This method takes a string and encodes it to a string that is ok to put in a URL.
Some similar questions:
URL is accessable with browser but still FileNotFoundException with URLConnection
URLConnection FileNotFoundException for non-standard HTTP port sources
URLConnection throwing FileNotFoundException
Wish you good luck.

It returns FileNotFoundException when the server response to HTTP request is code 404.
Check your URL.