Problems fetching URL in java with Jsoup - java

Edit: I have apparently solve the problem forcing the code getting the HTML. The problem I have is that randomly the HTML is not taken. To force that I have added:
int intento = 0;
while (document == null) {
intento++;
System.out.println("Intento número: " + intento);
document = getHtmlDocument(urlPage);
}
I am experiencing this random issue. Sometimes it gives me problems when fetching an URL an as it reaches to the timeout the program execution stops. The code:
public static int getStatusConnectionCode(String url) {
Response response = null;
try {
response = Jsoup.connect(url).userAgent("Mozilla/5.0").timeout(100000).ignoreHttpErrors(true).execute();
} catch (IOException ex) {
System.out.println("Excepción al obtener el Status Code: " + ex.getMessage());
}
return response.statusCode();
}
/**
* Con este método devuelvo un objeto de la clase Document con el contenido del
* HTML de la web que me permitirá parsearlo con los métodos de la librelia JSoup
* #param url
* #return Documento con el HTML
*/
public static Document getHtmlDocument(String url) {
Document doc = null;
try {
doc = Jsoup.connect(url).userAgent("Mozilla/5.0").timeout(100000).get();
} catch (IOException ex) {
System.out.println("Excepción al obtener el HTML de la página" + ex.getMessage());
}
return doc;
}
Should I use another method or increase the time out limit? The problem is that the program execution spends more or less 10 hours, and sometimes the problem happens in the URL number 500 another time in the 250...this is nonsense for me...if there is a problem in the link number 250, why if I run another time the program the problem happens in the link number 450 (for example)? I have been thinking that it could be internet problems but it's not.
The solution for another case is not solving my problem: Java JSoup error fetching URL
Thanks in advice.

Related

java.sql.SQLException: Column 'nombreGarcia' not found. why?

I cant connect my db with my java aplication and I really dont know whts wrong with it, comeone cant help me, please
try {
Connection miconexion = DriverManager.getConnection("jdbc:mysql://localhost:3306/concesionario","root","password");
// 2.PASO CREAR PASO STAMENT.
Statement miStatement = miconexion.createStatement();
//3.PASO EJECUTAR MY SQL.
ResultSet miresulset = miStatement.executeQuery("SELECT * FROM CLIENTES");
//4.PASO RECORRER EL RESULSET
while(miresulset.next()) {
System.out.println(miresulset.getString("dni" + miresulset.getString("nombre" + "" + miresulset.getString("apellido"))));
}
}catch(Exception e) {
System.out.println("NO CONECTA");
e.printStackTrace();
}
This line
System.out.println(miresulset.getString("dni" + miresulset.getString("nombre" + "" + miresulset.getString("apellido"))));
adds whatever is found as apellido to "nombre". The value happens to be "Garcia", hence the String of nombreGarcia in the error. This is as far as I can go with helping you without further information from your part.

JUnit test on recursive function (creating file and compare size)

I'm working on a project : copy a file and check if the size are equals. If not, delete file and redo it (number of retries is defined)
public boolean copieFichierAvecRetry(FileObject copieFichierFile, FileObject fichierACopier, int nbRetry, int currentNbRetry)
throws InterruptedException, IOException {
logger.logInfo("Deplacement du fichier " + fichierACopier.getName().getBaseName(),
"de " + fichierACopier.getParent().getName().getPath() + " vers "
+ copieFichierFile.getParent().getName().getPath());
copieFichierFile.copyFrom(fichierACopier, Selectors.SELECT_SELF);
boolean tailleOk = false;
// Si le flag de vérification est à true on vérifie que les fichiers
// copies ont la même taille
try {
tailleOk = verificationTailleCorrespondantes(copieFichierFile, fichierACopier);
if (!tailleOk && currentNbRetry <= nbRetry){
logger.logInfo("Erreur lors de la verification de la taille, essai n°" + currentNbRetry, null);
copieFichierFile.delete();
currentNbRetry++;
copieFichierAvecRetry(copieFichierFile, fichierACopier, nbRetry, currentNbRetry);
}
} catch (IOException e) {
logger.logWarn("Erreur lors de la verification de la taille : ", e.getMessage());
tailleOk = false;
}
return tailleOk;
}
Here is the unit test for the non-recursive function :
public void testCopieFichier()
throws IOException, InterruptedException, URISyntaxException, TransfertFichierException {
socleUtil.setNbTentativeMaxTransfert(1);
String nomFichierSource = "test123.txt";
String nomFichierDestination = "testDownloadSuccess.xml";
File fileOrigine = new File(getClass().getResource(SocleConstantes.SLASH).getFile());
String cheminFichierDistantOrigine = fileOrigine.getPath();
File fileDestination = new File(getClass().getResource(SocleConstantes.SLASH).toURI());
String cheminFichierDistantDestination = fileDestination.getPath() + FILE_SEPARATOR + "download";
assertTrue(socleUtil.copieFichier(
socleUtil.findFileLocal(cheminFichierDistantDestination + "/" + nomFichierDestination),
socleUtil.findFileLocal(cheminFichierDistantOrigine + "/" + nomFichierSource)));
assertTrue(fileDestination.exists());
}
As you can see in the code above, it will copy a file, check size and if it's OK then return true.
If it's false for 5 times (in the exemple) the function calls itself after deleting the file with wrong size.
verificationTailleCorrespondantes is the function to compare sizes.
tailleOk is true if both files are same size.
How should I test the recursivity of this function if it copies a file and never fail (which happens in production) ?
Thanks
In this case, I'd write the following scenarios:
the process success at its first iteration
the process fails at its (n-1)th iteration, success at its nth iteration with n < number of retries
the process fails at its nth iteration with n == number of retries
In order to do it, you'll need to mock your dependencies. Specially the one checking the file size. The mocking for the previous scenarios would be
file size check returns true. Assert that the check was run once and the result is valid
file size check returns false (n-1) times and true the nth time. Assert that the check was run n times and the result is valid
file size check returns false. Assert that the check was run number of retries times and the result is invalid

Can't succeed in linking an OpenGL program

Here are my (very simple) shaders:
protected String [] codeTransformateurSommets = {
"#version 400 core",
"void main(void) {",
"const vec4 vertices[3]=vec4[3](vec4(0.25,-0.25,0.5,1.0),",
" vec4(-0.25,-0.25,0.5,1.0),",
" vec4(0.25,0.25,0.5,1.0));",
"gl_position=vertices[gl_VertexID];",
"}"
};
protected String [] codeTransformateurFragments = {
"#version 400 core",
"out vec4 color;",
"void main(void) {",
"color=vec4(1.0,0.0,0.0,1.0);",
"}"
};
Here's how I try to get a program :
private int créeProgrammeOpenGL(){
IntBuffer résultatEditionLiens;
int transformateurSommets,transformateurFaces;
int programme;
transformateurSommets = chargeTransformateur(GL4.GL_VERTEX_SHADER,codeTransformateurSommets);
transformateurFaces = chargeTransformateur(GL4.GL_FRAGMENT_SHADER, codeTransformateurFragments);
programme = gl4.glCreateProgram();
gl4.glAttachShader(programme, transformateurSommets);
gl4.glAttachShader(programme, transformateurFaces);
gl4.glLinkProgram(programme);
if(programme!=0) {
résultatEditionLiens=IntBuffer.allocate(1);
gl4.glGetProgramiv(programme, GL4.GL_LINK_STATUS, résultatEditionLiens);
if (résultatEditionLiens.get(0) == 0) {
gl4.glDeleteProgram(programme);
throw new RuntimeException("Erreur à la création du programme!!");
} else {
gl4.glUseProgram(programme);
gl4.glDeleteShader(transformateurSommets);
gl4.glDeleteShader(transformateurFaces);
}
} else {
throw new RuntimeException("Erreur à la création du programme!!");
}
return(programme);
}
public static int chargeTransformateur(int type, String [] codeTransformateur){
int transformateur,i,j;
IntBuffer résultatCompilation;
transformateur = gl4.glCreateShader(type);
if(transformateur!=0) {
// création réussie : lui indiquer le code source
gl4.glShaderSource(transformateur,codeTransformateur.length,codeTransformateur,null);
// compiler le code source
gl4.glCompileShader(transformateur);
// récupérer le diagnostic de création du shader
byte [] infoLog=new byte[10000];
int [] taille=new int[100];
gl4.glGetShaderInfoLog(transformateur,1000,taille,0,infoLog,0);
// récupérer le résultat de la compilation
résultatCompilation=IntBuffer.allocate(1);
gl4.glGetShaderiv(transformateur,GL4.GL_COMPILE_STATUS,résultatCompilation);
if(résultatCompilation.get(0)==0) {
// la compilation a échoué!
throw new RuntimeException("Erreur à la compilation du shader!!"+"\r\n"+"\r\n"+codeTransformateur+"\r\n"+"\r\n"+infoLog.toString()+"\r\n"+"\r\n"+"\r\n");
}
} else {
// la création est un échec!
throw new RuntimeException("Erreur à la création du nom du shader!!"+"\r\n"+codeTransformateur);
}
return(transformateur);
}
And the result is a GL_LINK_STATUS set to false when glLinkProgram returns.
My investigations :
calling glGetShaderiv with parameter GL_COMPILE_STATUS returns true for both shaders.
calling glGetProgramiv function with parameter GL_ATTACHED_SHADERS returns 1 after the first attachment and 2 after the second.
Any idea about what I am doing wrong?
The major is is, that you've to set the source code to the shader object by glShaderSource rather than reading the source code string from a shader object by glGetShaderSource.
See Java Code Examples for com.jogamp.opengl.GL2.glShaderSource().
What you actually do, is to compile "empty" shader objects (of course with out any errors). But linking fails, because the vertex shader does not write to gl_Position (of course, because it is "empty").
If you want to create and initialize an array of vec4, then the correct syntax is:
(See Array constructors)
const vec4 vertices[3] = vec4[3](
vec4(0.25,-0.25,0.5,1.0),
vec4(-0.25,-0.25,0.5,1.0),
vec4(0.25,0.25,0.5,1.0));
Furthermore, OpenGL Shading Language (GLSL) is case sensitive. It is gl_Position rather than gl_position:
gl_position=vertices[gl_VertexID];
gl_Position = vertices[gl_VertexID];
I recommend to get the compile error messages by glGetShaderInfoLog. See Java Code Examples for org.lwjgl.opengl.GL20.glCompileShader()

How can I automate image upload to commercetools using JVM API

I'm a pretty new developer in CommerceTools and I've been working with this tool for just a few weeks.
At this moment I need to develop a process that should be able to upload all the images related to a product from a folder to commercetools using the JVM API.
I think the best way would be to recover the SKU (eg PROD001ABC) of each product from the CTP database and then use this string to locate in the given folder if there are images containing such SKU in the filename (PROD001ABC_front.jpg, PROD001ABC_side1.jpg, PROD001ABC_side2.jpg, etc.).
Once all the product images are located, I want to upload them to CommerceTools using the API.
As I've researched, I think I'd have to use the io.sphere.sdk.products.commands.ProductImageUploadCommand method, but I'm not sure how to get to that point.
I'm really lost.
Thanks so much for any help
Best regards.
Miguel
Basically what you need to do is to create an HttpClient and then use this client to execute the image upload command, to make things more concrete take a look at this test senario
here is the typical use of the commercetools JVM SDK for your purpose:
//create client
SphereClientConfig sphereClientConfig = SphereClientConfig.of( projectKey, clientId, clientSecret);
SphereClient client = SphereClient.of(sphereClientConfig, SphereClientFactory.of().createHttpClient(), SphereAccessTokenSupplier.ofConstantToken("accessToken"))
final ByIdVariantIdentifier identifier = product.getMasterData().getStaged().getMasterVariant().getIdentifier();
File imageFile = new File("Path to your image");
//create update commands
final ProductImageUploadCommand cmd1 = ProductImageUploadCommand
.ofVariantId(imageFile, identifier)
.withFilename("myProductImage1.gif")
.withStaged(true);
final ProductImageUploadCommand cmd2 = ProductImageUploadCommand
.ofVariantId(imageFile, identifier)
.withFilename("myProductImage2.gif")
.withStaged(true);
//update the product
final Product updatedProduct1 = client().executeBlocking(cmd1);
final Product updatedProduct = client().executeBlocking(cmd2);
//get the images
List<Image> images = updatedProduct.getMasterData().getStaged().getMasterVariant().getImages();
Hope this helps :)
Well, finally I have achieved it.
I used an attribute of my products (EAN) to locate the images in the path corresponding to "product type / EAN".
// Buscamos una carpeta con el nombre del EAN
final String pathCarpetaEan = rutaBaseLocal + "\\" + typeName + "\\" + vEan;
final File carpetaEAN = new File(pathCarpetaEan);
final File carpetaEanSubidas = new File(pathCarpetaEan + "\\subidas\\");
if (carpetaEAN.exists() && carpetaEAN.isDirectory()) {
// La carpeta existe. Buscamos imagenes dentro.
System.out.println("Encontrada la carpeta " + pathCarpetaEan);
File[] fileImages = carpetaEAN.listFiles();
for (File fileImagen : fileImages) {
if (fileImagen.isFile()) {
final String nomImagen = fileImagen.getName();
System.out.println("---\nNombre fichero: " + nomImagen);
if (nomImagen.startsWith(vEan)
&& (nomImagen.toLowerCase().endsWith(JPEG)
|| nomImagen.toLowerCase().endsWith(PNG)
|| nomImagen.toLowerCase().endsWith(GIF))) {
System.out.println("El nombre de archivo contiene el EAN: " + vEan);
System.out.println("Y se trata de una imagen");
// A partir de aqui realizamos la subida de las imagenes para la variante concreta.
final ProductImageUploadCommand cmd = ProductImageUploadCommand
.ofVariantId(fileImagen, identificador)
.withFilename(nomImagen)
.withStaged(true);
final Product updatedProduct = client.executeBlocking(cmd);
System.out.println("Uploaded your image (" + nomImagen + ") and added to the masterVariant");
System.out.println("Producto actualizado: " + updatedProduct.toString());
nUploadedImages++;
}
}
}
}
After uploading the images, I move them to another subfolder "uploaded".
Thank you very much for your help.

Recieving json with html tags fails in PHP

I have a website that sends and receives string in JSON format from my Java REST server with jersey. Everything works fine until I'm trying to receive a json object with html tags.
A println on my java server tells me that this data has ben sent:
data sent: {"text": "Wij zijn Pixel Apps, ook wel bekend als Groep 6.<br />
Samen met onze 6 groepsleden verzorgen wij het reilen en zijlen op Ford Lommel Proving Grounds.<br />
<br />
<b>Korte inleiding</b><br />
<p>Onze taak bestaat er uit een functionele applicatie te maken binnen Windows 8. De app bestaat er uit de chauffeurs te begeleiden op hun testritten.<br />De chauffeurs worden onder andere geholpen bij het bekijken van hun routineplan, het bijhouden van notities en het overzetten van de resultaten naar het hoofdgebouw.</p>
<b>Bijkomende hoort natuurlijk het onderhouden van deze website.</b>
<p>Zoals u kan zien vind u hierboven het navigatiemenu.<br />
Voor meer informatie over ons project kan u terecht bij <i>Over ons</i><br />
Wenst u contact op te nemen? U kan zich wenden naar het tabblad <i>Contact</i><br />
Indien u meer over de individuele groepsleden wil weten kan u terecht bij <i>Leden</i><br />
Als u meer informatie wenst over ons project, gelieve contact op te nemen met ons en wij verzorgen uw verzoek.</p>
<b>Happy browsing!</b>"}
It's basically a simple json with one variable "text" and as content some HTML formatted content. I've googled my issue and it seems that this should work fine.
Here's my java GET method that fails to send json with html tags in it's content:
#GET
#Path("gettext")
#Produces("application/json")
public String getJson(#QueryParam("id") String id, #QueryParam("taalcode") String taalcode) {
Connectie c = new Connectie();
try
{
c.openConnectie();
String content = c.getCms(id, taalcode);
if (content == null || content.equals("")) {
content = "{ \"text\" : \"Geen tekst gevonden.\" }";
}
System.out.println("data send: "+content);
return content;
}
catch(Exception e)
{
System.out.println("data send: { \"text\" : \"Server error, sorry.\" }");
return "{ \"text\" : \"Server error, sorry.\" }";
}
}
My put method successfully receives a json with html tags in it's content.
Here's how I receive my json objects in PHP (which again works if no html tags are present):
public function getCMS($id) {
$taalcode = '';
if($this->session->userdata('language') == 'nederlands') {
$taalcode = 'NL';
} else {
$taalcode = 'EN';
}
$curl_instance = curl_init();
curl_setopt($curl_instance, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl_instance, CURLOPT_URL, 'http://192.168.0.251:8084/Groep1/webresources/cmspost/gettext?id='.$id.'&taalcode='.$taalcode);
try {
$data = json_decode(curl_exec($curl_instance), true);
if ($data == null) {
$data['text'] = "Altough I set a string in my java get method if it's null, this message is always printed";
}
return $data;
} catch (HttpException $ex) {
$data['text'] = $ex;
return $data;
}
}
In PHP I test if ($data == null) which is always true, even though I set a string manually in my GET method if appears to be null before sending the string.
What am I doing wrong?
The problem is not with HTML. The problem is that JSON does not allow multi-line strings. If you remove the line breaks, your JSON works fine.
NB that you really should use a JSON library for building JSON, rather than doing it yourself, because it will deal with this kind of issue.

Categories