We Keep Coding

I'm trying to write a array bytes in a file but i can not. (I can create this file in FileSystem, and the conection is working). Any help is grateful

The main errors are:
File /test/1.txt could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
Exception in createBlockOutputStream.
Also i try to write a String and its imposible.
public void establece_connexion() {
try {
//System.setProperty("HADOOP_USER_NAME", "hduser"); //por si acaso lo necesitamos.
//UserGroupInformation ugi = UserGroupInformation.createRemoteUser(usuario_remoto);
Configuration conf = new Configuration();
conf.set("hadoop.job.ugi", usuario_remoto);
conf.set("fs.defaultFS", namenode); // namenode ="hdfs://" + ip + ":" + puerto
hdfsFileSystem = FileSystem.get(conf);
//hdfsFileSystem = FileSystem.get(new URI(namenode), new Configuration());
System.out.println("Se declara HDFS como sistema de archivos subyacente");
} catch (IOException e) {
e.printStackTrace();
System.out.println("Error en la conexión");
}
}
public void write() {
int numero = 1;
while (numero <= get_numero_ficheros_totales()-1) {
try {
System.out.println("Voy a empezar a escribir DE NUEVO");
//Path workingDir = hdfsFileSystem.getWorkingDirectory();
//Path archivo = new Path(hdfsFileSystem.getWorkingDirectory() + "/" + hdfsPath + "/" + numero + ".txt"); //creamos el nuevo fichero//
Path archivo = new Path(namenode + "//test//" + numero + ".txt");
System.out.println("ESTOY AQUI YA ");
//Hay una comprobacion de si si existe el ficehro que yo he creado
if (hdfsFileSystem.exists(archivo)) {
System.out.println("El fichero " + numero + " ya existe, no lo voy a crear de nuevo");
} else { //una vez que comprobamos que no existe el archivo, o ya esta borrado el archivo lo creamos o lo cremos de nuevo.
System.out.println("no existe el archivo" + numero);
FSDataOutputStream fsdata = hdfsFileSystem.create(archivo);
try {
byte[] arrayBytes = array_relleno();
//assert false;
fsdata.write(arrayBytes,0, arrayBytes.length); //esto sería sin escritura en zip//
//fsdata.writeUTF("hello");
//fsdata.writeUTF("hello");
System.out.println("Voy a empezar a escribir DE NUEVO");
System.out.println(fsdata);
fsdata.close();
} catch (Exception e) {
System.out.println("No he podido escribir en el fichero");
}
}
} catch(IOException e){
e.printStackTrace();
}
numero++;
}
}
Errors:
Se declara HDFS como sistema de archivos subyacente
Voy a empezar a escribir DE NUEVO
ESTOY AQUI YA
no existe el archivo1
Voy a empezar a escribir DE NUEVO
org.apache.hadoop.hdfs.client.HdfsDataOutputStream#6ccdb29f
2019-08-26 09:45:21,849 INFO [Thread-4] hdfs.DFSClient (DFSOutputStream.java:createBlockOutputStream(1368)) - Exception in createBlockOutputStream
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1533)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1309)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1262)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:448)
2019-08-26 09:45:21,854 INFO [Thread-4] hdfs.DFSClient (DFSOutputStream.java:nextBlockOutputStream(1265)) - Abandoning BP-328389572-127.0.1.1-1566466459521:blk_1073742162_1338
2019-08-26 09:45:21,916 INFO [Thread-4] hdfs.DFSClient (DFSOutputStream.java:nextBlockOutputStream(1269)) - Excluding datanode DatanodeInfoWithStorage[127.0.0.1:50010,DS-9a5d1ada-fad9-4916-82c3-3c7125a6907a,DISK]
2019-08-26 09:45:21,925 WARN [Thread-4] hdfs.DFSClient (DFSOutputStream.java:run(557)) - DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /test/1.txt could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1576)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3107)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3031)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:725)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045)
at org.apache.hadoop.ipc.Client.call(Client.java:1475)
at org.apache.hadoop.ipc.Client.call(Client.java:1412)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy10.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:418)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy11.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1455)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1251)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:448)
No he podido escribir en el fichero

I think I was allowed to create the files but not write to them because I think I accessed the namenode but when I accessed the datanode I gave the error

I have got a answer, the problem is solved.
I've been trying to connect to a cluster to another linux (ubuntu 18.10) computer where hadoop was installed. i acess whit mi windows computer (all in the same network).
In file core-site.xml I have changed "localhost" to 0.0.0.0 for access from mi other machine (windows). there was the problem. i change the ip to 192.168.3.37 (ip where are you dating) and it`s working fine. and then reestart namenode and datanode.
old-dated---------------:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://0.0.0.0:8020</value>
<description>Nombre del filesystem por defecto.</description>
</property>
</configuration>
up-dated---------------:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://192.168.3.37:8020</value>
<description>Nombre del filesystem por defecto.</description>
</property>
</configuration>
At the beginning of the whole put localhost, but that could not remain to have to be accessible from another machine

Read specific data from XML using Java (DOM)

I want to extrat some data from an invoice (XML) using Java. I'm using the example on this link (https://www.mkyong.com/java/how-to-read-xml-file-in-java-dom-parser/)
The problem is my XML doesn't look like the one in the example, here is mine:
<?xml version="1.0" encoding="UTF-8"?>
<cfdi:Comprobante xmlns:cfdi="http://www.sat.gob.mx/cfd/3" xmlns:bfa2="http://www.buzonfiscal.com/ns/addenda/bf/2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" NumCtaPago="4007" LugarExpedicion="CUAJIMALPA DE MORELOS,CIUDAD DE MEXICO" metodoDePago="99" tipoDeComprobante="ingreso" total="2490" subTotal="2146.55" fecha="2016-12-14T10:23:08" folio="57318" serie="BT" version="3.2" xsi:schemaLocation="http://www.sat.gob.mx/cfd/3 http://www.sat.gob.mx/sitio_internet/cfd/3/cfdv32.xsd http://www.buzonfiscal.com/ns/addenda/bf/2 http://www.buzonfiscal.com/schema/xsd/Addenda_BF_v20.xsd">
<cfdi:Emisor nombre="BEST BUY STORES S. DE R.L. DE C.V." rfc="BBS070606D33">
<cfdi:DomicilioFiscal codigoPostal="05348" pais="MEXICO" estado="CIUDAD DE MEXICO" municipio="CUAJIMALPA DE MORELOS" referencia="EDIFICIO CENTURY PLAZA" colonia="SANTA FE CUAJIMALPA" noInterior="PISO 2 OFNA 202 Y 203 P3 Y P4" noExterior="440" calle="AV. SANTA FE" />
<cfdi:ExpedidoEn codigoPostal="05348" pais="MEXICO" estado="CIUDAD DE MEXICO" municipio="CUAJIMALPA DE MORELOS" colonia="SANTA FE" calle="AVENIDA SANTA FE NO. 440 PISO 2 (OFICINA 203 Y 204),PISO 3 Y PISO 4" />
<cfdi:RegimenFiscal Regimen="Regimen General de Ley Personas Morales" />
</cfdi:Emisor>
<cfdi:Receptor nombre="VIALCOMA S.A DE C.V" rfc="VIA8011248W0">
<cfdi:Domicilio codigoPostal="54080" pais="México" estado="Estado de México" municipio="Tlalnepantla de Baz" referencia="VIALCOMA S.A. de C.V." localidad="Tlalnepantla de Baz" colonia="EL MIRADOR" noExterior="107" calle="AV. GUSTAVO BAZ " />
</cfdi:Receptor>
<cfdi:Conceptos>
<cfdi:Concepto importe="2146.55" valorUnitario="2146.55" descripcion="PANASONIC PHONE KX-PRD260MEB" noIdentificacion="000000001000189023" unidad="PZA" cantidad="1" />
</cfdi:Conceptos>
<cfdi:Impuestos totalImpuestosTrasladados="343.45">
<cfdi:Traslados>
<cfdi:Traslado importe="343.45" tasa="16.0" impuesto="IVA" />
</cfdi:Traslados>
</cfdi:Impuestos>
<cfdi:Complemento>
<tfd:TimbreFiscalDigital xmlns:tfd="http://www.sat.gob.mx/TimbreFiscalDigital" xsi:schemaLocation="http://www.sat.gob.mx/TimbreFiscalDigital http://www.sat.gob.mx/sitio_internet/cfd/TimbreFiscalDigital/TimbreFiscalDigital.xsd" UUID="2e36935c-4948-4f7b-9b29-477546aa2922" version="1.0" />
</cfdi:Complemento>
<cfdi:Addenda>
<bfa2:AddendaBuzonFiscal version="2.0">
<bfa2:Receptor email="" />
<bfa2:TipoDocumento descripcion="Factura" nombreCorto="FAC" />
<bfa2:CFD totalConLetra="Dos mil cuatrocientos noventa pesos 00/100 M.N." fechaEnvio="2016-12-12T10:42:15-06:00" />
<bfa2:Extra valor="02028001014520161212" atributo="numTicket" />
</bfa2:AddendaBuzonFiscal>
</cfdi:Addenda>
</cfdi:Comprobante>
I want to get (print on console) the data on "folio" "serie" and "UUID"
Can someone help me please?

The following should give your desire output.This example is based on DOM parser as you have provide.
System.out.println("Root element :" + doc.getDocumentElement().getNodeName());
NodeList nList = doc.getElementsByTagName("cfdi:Comprobante");
org.w3c.dom.Node nNode = nList.item(0);
System.out.println("folio: " + ((Element) nNode).getAttribute("folio"));
System.out.println("serie: " + ((Element) nNode).getAttribute("serie"));
NodeList list = doc.getElementsByTagName("cfdi:Complemento");
for (int i = 0; i < 1; i++) {
NodeList child = list.item(i).getChildNodes();
for (int j = 0; j < child.getLength(); j++) {
if (child.item(j).getNodeName().equals("tfd:TimbreFiscalDigital")) {
String uuid = ((Element)(child.item(j))).getAttribute("UUID");
System.out.println("UUID : " + uuid);
}
}
}
console output:
Root element :cfdi:Comprobante
----------------------------
folio: 57318
serie: BT
UUID : 2e36935c-4948-4f7b-9b29-477546aa2922

Problems fetching URL in java with Jsoup

Edit: I have apparently solve the problem forcing the code getting the HTML. The problem I have is that randomly the HTML is not taken. To force that I have added:
int intento = 0;
while (document == null) {
intento++;
System.out.println("Intento número: " + intento);
document = getHtmlDocument(urlPage);
}
I am experiencing this random issue. Sometimes it gives me problems when fetching an URL an as it reaches to the timeout the program execution stops. The code:
public static int getStatusConnectionCode(String url) {
Response response = null;
try {
response = Jsoup.connect(url).userAgent("Mozilla/5.0").timeout(100000).ignoreHttpErrors(true).execute();
} catch (IOException ex) {
System.out.println("Excepción al obtener el Status Code: " + ex.getMessage());
}
return response.statusCode();
}
/**
* Con este método devuelvo un objeto de la clase Document con el contenido del
* HTML de la web que me permitirá parsearlo con los métodos de la librelia JSoup
* #param url
* #return Documento con el HTML
*/
public static Document getHtmlDocument(String url) {
Document doc = null;
try {
doc = Jsoup.connect(url).userAgent("Mozilla/5.0").timeout(100000).get();
} catch (IOException ex) {
System.out.println("Excepción al obtener el HTML de la página" + ex.getMessage());
}
return doc;
}
Should I use another method or increase the time out limit? The problem is that the program execution spends more or less 10 hours, and sometimes the problem happens in the URL number 500 another time in the 250...this is nonsense for me...if there is a problem in the link number 250, why if I run another time the program the problem happens in the link number 450 (for example)? I have been thinking that it could be internet problems but it's not.
The solution for another case is not solving my problem: Java JSoup error fetching URL
Thanks in advice.

Selenium getText() method does not get correct text with diacritics

I am trying to get text from a field that contains diacritics like: ă,ă-Ă-â-Â-î-Î-ş-Ş-ţ-Ţ
My method for get text looks like this:
public void invalidEmailValidation(){
String expectedInner = LoginPage.invalidMailMessage(driver).getAttribute("innerText");
String expectedText = LoginPage.invalidMailMessage(driver).getText();
System.out.print("EXPECTED1: " + expectedInner + "\n" +
"EXPECTED2: " + expectedText );
Assert.assertEquals("Expcted Inner: Adresa de email nu este validă",
expectedInner, "Adresa de email nu este validă");
Assert.assertEquals("Expcted text: Adresa de email nu este validă",
expectedText, "Adresa de email nu este validă");
The output from the method is here:
I mention that my project have UTF-8 encoding
Also UTF-8 encoding is added in pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0
Other informations:
IDE: IntellijIdea 14.1.4 (comunity version)
OS: Windows 7 (english language as default)
I already checked the following questions from StackOverflow:
Cannot verify text with diacritics
Empty string

Recieving json with html tags fails in PHP

I have a website that sends and receives string in JSON format from my Java REST server with jersey. Everything works fine until I'm trying to receive a json object with html tags.
A println on my java server tells me that this data has ben sent:
data sent: {"text": "Wij zijn Pixel Apps, ook wel bekend als Groep 6.<br />
Samen met onze 6 groepsleden verzorgen wij het reilen en zijlen op Ford Lommel Proving Grounds.<br />
<br />
<b>Korte inleiding</b><br />
<p>Onze taak bestaat er uit een functionele applicatie te maken binnen Windows 8. De app bestaat er uit de chauffeurs te begeleiden op hun testritten.<br />De chauffeurs worden onder andere geholpen bij het bekijken van hun routineplan, het bijhouden van notities en het overzetten van de resultaten naar het hoofdgebouw.</p>
<b>Bijkomende hoort natuurlijk het onderhouden van deze website.</b>
<p>Zoals u kan zien vind u hierboven het navigatiemenu.<br />
Voor meer informatie over ons project kan u terecht bij <i>Over ons</i><br />
Wenst u contact op te nemen? U kan zich wenden naar het tabblad <i>Contact</i><br />
Indien u meer over de individuele groepsleden wil weten kan u terecht bij <i>Leden</i><br />
Als u meer informatie wenst over ons project, gelieve contact op te nemen met ons en wij verzorgen uw verzoek.</p>
<b>Happy browsing!</b>"}
It's basically a simple json with one variable "text" and as content some HTML formatted content. I've googled my issue and it seems that this should work fine.
Here's my java GET method that fails to send json with html tags in it's content:
#GET
#Path("gettext")
#Produces("application/json")
public String getJson(#QueryParam("id") String id, #QueryParam("taalcode") String taalcode) {
Connectie c = new Connectie();
try
{
c.openConnectie();
String content = c.getCms(id, taalcode);
if (content == null || content.equals("")) {
content = "{ \"text\" : \"Geen tekst gevonden.\" }";
}
System.out.println("data send: "+content);
return content;
}
catch(Exception e)
{
System.out.println("data send: { \"text\" : \"Server error, sorry.\" }");
return "{ \"text\" : \"Server error, sorry.\" }";
}
}
My put method successfully receives a json with html tags in it's content.
Here's how I receive my json objects in PHP (which again works if no html tags are present):
public function getCMS($id) {
$taalcode = '';
if($this->session->userdata('language') == 'nederlands') {
$taalcode = 'NL';
} else {
$taalcode = 'EN';
}
$curl_instance = curl_init();
curl_setopt($curl_instance, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl_instance, CURLOPT_URL, 'http://192.168.0.251:8084/Groep1/webresources/cmspost/gettext?id='.$id.'&taalcode='.$taalcode);
try {
$data = json_decode(curl_exec($curl_instance), true);
if ($data == null) {
$data['text'] = "Altough I set a string in my java get method if it's null, this message is always printed";
}
return $data;
} catch (HttpException $ex) {
$data['text'] = $ex;
return $data;
}
}
In PHP I test if ($data == null) which is always true, even though I set a string manually in my GET method if appears to be null before sending the string.
What am I doing wrong?

The problem is not with HTML. The problem is that JSON does not allow multi-line strings. If you remove the line breaks, your JSON works fine.
NB that you really should use a JSON library for building JSON, rather than doing it yourself, because it will deal with this kind of issue.