Informix db clob with utf-8 characters stores incomplete data - java

I am facing a problem with my Informix db while storing CLOB data with UTF-8 charset data.
The regular varchar datatype with UTF-8 characters is storing without any issue; however, when I store the data as a CLOB and data (in my case XML) contains utf-8 characters, only truncated XML is stored.
Example Characters: german umlaut
Informix db character encoding is set to utf-8.
Below is my hbm file
<?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
<hibernate-mapping package="com.test.adapter.persistence.bean">
<class name="MediaLayoutConfigPojo" table="medialayoutconfig">
<id name="dbId" type="int" column="id">
<generator class="native" />
</id>
<property name="layoutxml" type="clob" column="layoutxml" />
<property name="name" type="string" column="name" />
<property name="description" type="string" column="description" />
<property name="layouttype" type="string" column="layouttype" />
</class>
</hibernate-mapping>
SQL driver class is
<property name="connection.driver_class">com.informix.jdbc.IfxDriver</property>
<property name="dialect">org.hibernate.dialect.InformixDialect</property>
I am using the ifxjdbc-3.50.JC9.jar JDBC driver (unfortunately, I'm stuck with it; upgrading it may cause additional breakages since it's a legacy application).
Hibernate version used is: hibernate-core-3.6.3.Final.jar
I went through this link https://hibernate.atlassian.net/browse/HHH-6127. Not sure If I am hitting the same issue in driver class.
My table definition and sample data I am trying store and what exactly stored in given below:
Here is my table definition:
Column name Type Nulls
id serial no
layouttype varchar(32,0) no
name lvarchar(256) no
description lvarchar(512) yes
layoutxml clob no
This the data I am trying to insert into CLOB layoutxml:
<MediaPropertiesLayout><name>Система</name><description></description><header><entry><showInPopOver/><displayName>Система</displayName><mediaProperty>callVariable1</mediaProperty><uiEditable>false</uiEditable></entry></header><column></column><column></column><uri></uri></MediaPropertiesLayout>
Actual data stored in layoutxml (CLOB)
<MediaPropertiesLayout><name>Система</name><description></description><header><entry><showInPopOver/><displayName>Система</displayName><mediaProperty>callVariable1</mediaProperty><uiEditable>false</uiEditable></entry></header><column></column><column></column><uri>
i.e. Closing tags for uri and MediaPropertiesLayout is truncated. This will vary depending upon the UTF-8 characters I use. If I use larger string instead of "Система", then truncation will be at different place.
Any pointer will be great help.

Are you sure you are passing a correct UTF8 value?
Usually when you get truncation is because the encoding is wrong.
I tried inserting the UTF8 string 'Füße' (46 C3BC C39F 65) with a simple Java console app and it behaves as expected.
D:\Infx\ids1410>chcp 65001
Active code page: 65001
D:\Infx\ids1410>java com.informix.jdbc.Version
IBM Informix JDBC Driver Version 3.50.JC9X3
D:\Infx\ids1410>javac -encoding utf8 clob.java
D:\Infx\ids1410>od -tcx1 clob.txt
0000000000 F e
46 C3 BC C3 9F 65
0000000006
D:\Infx\ids1410>java -Dfile.encoding=UTF-8 clob
--Füße--Füße--
D:\Infx\ids1410>oncheck -pp utf8:t1 256
addr stamp chksum nslots flag type frptr frcnt next prev
2:4566 36224780 acf0 1 1 DATA 106 3982 0 0
slot ptr len flg
1 24 82 0
slot 1:
0: 6 46 c3 bc c3 9f 65 0 0 0 1 0 0 0 d9 c8 .FC<C.e.......YH
16: b7 a6 3 0 0 0 3 0 0 0 3 0 0 0 a1 f8 7&............!x
32: 10 61 0 0 0 0 1 0 0 0 0 0 0 0 0 0 .a..............
48: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ................
64: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ................
80: 0 0 ................
D:\Infx\ids1410>dbaccess utf8 -
Database selected.
> unload to t1.unl select * from t1;
1 row(s) unloaded.
> !od -x t1.unl
0000000000 C346 C3BC 659F 307C 362C 632C 6F6C 3162
0000000020 3031 2E66 6138 7C31 000A
0000000031
> !cat t1.unl
Füße|0,6,clob110f.8a1|
> !od -x clob110f.8a1
0000000000 C346 C3BC 659F
0000000006
> !cat clob110f.8a1
Füße>
>
I don't have 3.50.JC9 at hand, but 3.50.JC8 and 3.50.JC9X3 did the same so I'm sure the 'vanilla' 3.50.JC9 will work too.
Maybe you can try the console app with your xml data (make sure it is in UTF8) and see if it still gets corrupted or not.
// ----- clob.java -----
import java.sql.*;
import java.io.*;
import com.informix.jdbc.*;
public class clob {
public static void main(String[] args) {
Connection con = null;
try {
InputStreamReader isr = new InputStreamReader( System.in );
BufferedReader stdin = new BufferedReader( isr );
Class.forName("com.informix.jdbc.IfxDriver");
con = DriverManager.getConnection("jdbc:informix-sqli://420ito:9091/utf8:INFORMIXSERVER=ids1410;user=informix;password=mypassw;DB_LOCALE=en_US.utf8;SQLIDEBUG=pp");
System.out.println ("");
Statement stm1 = con.createStatement();
try {
stm1.executeUpdate ("drop table t1;");
} catch (Exception ex) {}
stm1.executeUpdate ("create table t1 (c1 varchar(20), c2 clob);");
PreparedStatement ps = con.prepareStatement("insert into t1 values ('Füße',?);");
FileInputStream fis = new FileInputStream("clob.txt");
BufferedInputStream dataStream = new BufferedInputStream(fis);
ps.setAsciiStream(1, dataStream, dataStream.available());
ps.execute();
ps.close();
PreparedStatement ps2 = con.prepareStatement("select * from t1;");
IfxResultSet resultSet = (IfxResultSet) ps2.executeQuery();
while (resultSet.next()) {
System.out.format("--%s--%s--\n",resultSet.getString(1),resultSet.getString(2));
try {
} catch ( Exception ex ) {};
}
resultSet.releaseBlob();
ps2.close();
((IfxSqliConnect) con).releaseBlob();
} catch (Exception ex) {
ex.printStackTrace();
if (ex instanceof SQLException) {
SQLException sqlex = (SQLException) ex;
System.out.println ("Message: " + sqlex.getMessage());
}
}
}
}
// ----- clob.java -----
If it works the issue maybe in how Hibernate handles that data.

Issue still exists in creating Clob in Hibernate. Somehow, Hibernate not able to calculate the string length based on UTF-8 characters. When you have these kind of special character, you need layoutXML.getBytes("utf-8").length instead layoutXML.length(). This difference is actually causing the truncation.
However, I skipped the entire conversion overhead all together, just by declaring 'layoutxml' as String in POJO instead of java.sql.Clob. For this I upgraded the hibernate version to 5.6.1.Final.
With 3.5.2-Final hibernate verison, even this approach doesn’t work.
Note: Declaring CLOB entries as Strings in POJO has its own disadvantages related to performance.

Related

How to retrieve host, port, sid, user and password informations in java.sql.Connection

I have an Oracle connection (in a java.sql.Connection object) that is mounted from a properties file (ApplicationResources.properties).
How do I capture information about host, port, sid, user and password? (I need this information to call a specific database function through my java application)
Below my code:
private void conectaBanco() {
ServiceLocator sl = new ServiceLocator();
InitialContext ic = null;
DataSource dataSource = null;
try {
if (conn == null) {
ic = new InitialContext();
dataSource = (DataSource) ic.lookup(sl.getInfo().getString("JNDI"));
conn = dataSource.getConnection();
conn.setAutoCommit( true );
// I need host, port, sid and password information
System.out.println("USER " + conn.getMetaData().getUserName());
}
} catch (SQLException e) {
logger.error("Erro ao tentar abrir conexao com o banco de dados.", e);
} catch (NamingException e) {
logger.error("Erro ao tentar abrir conexao com o banco de dados.", e);
}
if(conn != null)
rastrear.definePredicate(conn);
}
SESSION SID:
SELECT sys_context('USERENV', 'SID') FROM DUAL;
ORACLE_SID:
SELECT sys_context('userenv','instance_name') FROM dual;
HOST (Database machine) :
SELECT UTL_INADDR.get_host_name FROM dual;
Password: See http://www.dba-oracle.com/t_password_storage.htm
CLIENT SIDE Port:
Select port from v$session;
SERVER OS PID of Oracle server process connected to client process
SELECT p.spid
FROM v$process p, v$session s
WHERE s.paddr = p.addr and
sys_context('USERENV', 'SID') = s.sid;
Server side port:
This is really difficult since the server process port is mapped to a different port then the initial - for example port 12102 in listener.ora/tnsnames.ora get mapped by listener to a arbitrary free one (by the way: thats the reason why often firewalls need to be shutdown and while connections cannot get after mapping through firewall. This can be fixed but this is another story)
Below code
grants required privileges
defines a java stored function "Util.RunThis" which executes on database server a OS command passed into function and return string.
"Util.RunThis" is mapped to PL/SQL function "RUN_CMD"
"GET_PORT" is a PL/SQL function returning a numeric value of the used Port of the database server process connected to client session. Inside GET_PORT the SELECT is used to determine database server process pid replacing [SPID] in command below
/usr/sbin/lsof -Pan -p [SPID] -i
finally invoking a simple select we get the Port of database server process attached to current session
connect / as sysdba
grant select on sys.v_$process to scott;
grant select on sys.v_$session to scott;
begin
dbms_java.grant_permission
('SCOTT',
'SYS:java.io.FilePermission',
'<<ALL FILES>>',
'execute');
end;
/
connect scott/tiger#foo.com
create or replace and compile java source named "Util"
as
import java.io.*;
import java.lang.*;
import java.nio.charset.Charset;
import java.nio.*;
public class Util extends Object
{
public static String RunThis(String args)
{
Runtime rt = Runtime.getRuntime();
String rc = args;
try
{
Process p = rt.exec(args);
int bufSize = 4096;
BufferedInputStream bis =
new BufferedInputStream(p.getInputStream(), bufSize);
int len;
byte buffer[] = new byte[bufSize];
// Echo back what the program spit out
while ((len = bis.read(buffer, 0, bufSize)) != -1)
{
String xxx = new String(buffer, Charset.forName("UTF-8"));
rc = rc + xxx;
}
p.waitFor();
rc = rc + "ABC";
}
catch (Exception e)
{
e.printStackTrace();
rc = "Exception!!!" ;
}
finally
{
return rc;
}
}
}
/
create or replace function RUN_CMD( p_cmd in varchar2) return VARCHAR2
as language java
name 'Util.RunThis(java.lang.String) return java.lang.String';
/
create or replace function GET_PORT return number
as
SPID NUMBER;
retval varchar2(32000);
y varchar2(1024);
cmd VARCHAR2(256);
begin
SELECT
p.spid
INTO
SPID
FROM
sys.v_$process p,
sys.v_$session s
WHERE
s.paddr = p.addr and
sys_context('USERENV', 'SID') = s.sid;
cmd := '/usr/sbin/lsof -Pan -p [SPID] -i';
/* raw string data returned from Shell executing cmd */
retval := run_cmd(replace(cmd,'[SPID]', SPID));
/* get last occurance of : marking redirected port */
y := substr(retval,INSTR(retval,':', -1)+1,1024);
/* return the numeric port by stripping info like " (ESTABLISHED)" */
return to_number(substr(y,1,INSTR(y, ' ')-1));
end;
/
show errors
select get_port from dual;
/*-------------------- OUTPUT -------------------------- */
SQL> connect / as sysdba
grant select on sys.v_$process to scott;
grant select on sys.v_$session to scott;
begin
dbms_java.grant_permission
('SCOTT',
'SYS:java.io.FilePermission',
'<<ALL FILES>>',
'execute');
end;
/Connected.
SQL> SQL>
Grant succeeded.
SQL> SQL>
Grant succeeded.
SQL> SQL> 2 3 4 5 6 7 8
PL/SQL procedure successfully completed.
SQL> connect scott/tiger#foo.com
Connected.
SQL> create or replace and compile java source named "Util"
2 as
import java.io.*;
import java.lang.*;
import java.nio.charset.Charset;
3 4 5 6 import java.nio.*;
7 public class Util extends Object
8 {
9 public static String RunThis(String args)
{
Runtime rt = Runtime.getRuntime();
10 11 12 String rc = args;
13
14 try
15 {
16
17 Process p = rt.exec(args);
18 19 int bufSize = 4096;
20 BufferedInputStream bis =
21 new BufferedInputStream(p.getInputStream(), bufSize);
22 int len;
byte buffer[] = new byte[bufSize];
23 24
// Echo back what the program spit out
while ((len = bis.read(buffer, 0, bufSize)) != -1)
25 26 27 {
28 String xxx = new String(buffer, Charset.forName("UTF-8"));
29 rc = rc + xxx;
30 }
p.waitFor();
31 32
rc = rc + "ABC";
33 34 35 }
36 catch (Exception e)
37 {
38 e.printStackTrace();
39 rc = "Exception!!!" ;
40 }
41
42 finally
43 {
44 return rc;
45 }
}
46 47 }
/ 48
Java created.
SQL> create or replace function RUN_CMD( p_cmd in varchar2) return VARCHAR2
as language java
name 'Util.RunThis(java.lang.String) return java.lang.String';
/ 2 3 4
Function created.
SQL> create or replace function GET_PORT return number
as
SPID NUMBER;
retval varchar2(32000);
2 3 4 5 y varchar2(1024);
cmd VARCHAR2(256);
begin
6 7 8 SELECT
p.spid
INTO
9 10 11 SPID
FROM
sys.v_$process p,
sys.v_$session s
WHERE
12 13 14 15 16 s.paddr = p.addr and
sys_context('USERENV', 'SID') = s.sid;
cmd := '/usr/sbin/lsof -Pan -p [SPID] -i';
17 18 19 20 21 /* raw string data returned from Shell executing cmd */
retval := run_cmd(replace(cmd,'[SPID]', SPID));
22 23 24 /* get last occurance of : marking redirected port */
y := substr(retval,INSTR(retval,':', -1)+1,1024);
/* return the numeric port by stripping info like " (ESTABLISHED)" */
25 26 27 28 return to_number(substr(y,1,INSTR(y, ' ')-1));
end;
/
show errors 29 30
Function created.
SQL>
No errors.
SQL> select get_port from dual;
GET_PORT
----------
36586
I solve this problem adding on my Weblogic configuration JDBC the additional parameters (host, user, port, sid - the password is returned automaticaly).
And I got this values throw this code below:
import weblogic.jdbc.wrapper.PoolConnection;
import java.util.Properties;
(...)
public Properties retornaPropriedadesConexao(){
PoolConnection pc = (PoolConnection)conn;
Properties p = pc.getConnectionEnv().getDriverProperties();
return p;
}

searching and replacing content from one file to another file using a keyword using python or java

Locked. There are disputes about this question’s content being resolved at this time. It is not currently accepting new answers or interactions.
I have 2 files which contains some data like this!!
File 1 contains:
/begin MENT AE0DAQ0O41 ""
ECU_ADDRESS 0x8111DSCC
ECU_ADDRESS_EXTENSION 0x0
/begin IF_DATA CAN_EXT
120
LINK_MAP "AE0DAQ0O41" 0x8111DSCC 0x0 0 0x2 1 0x2F 0x1
DISPLAY 0 0 655
/end IF_DATA
SYMBOL_LINK "AE0DAQ0O41" 0
/end MENT
File 2 contains:
name value line keyword
.data 80008114+000005 AE0DAQ0O43
.data 80008116+000005 AE0DAQ0O41
.data 80008118+000005 EA0DAQ0O45
.data 8000811a+000005 AE0DAF0O89
Now what we need to do is take a keyword AE0DAQ0O41 and need to search in the next file.
It has some value before the keyword, so we need to take that value 80008116 and need to replace it in
ECU_ADDRESS 0x8111DSCC and also LINK_MAP AE0DAQ0O41 0x8111DSCC 0x0 0 0x2 1 0x2F 0x1 for (0x8111DSCC it needs to be 0x80008116) and save it to FILE 1.
FILE 1 is to be saved like this :
/begin MENT AE0DAQ0O41 ""
ECU_ADDRESS 0x80008116
ECU_ADDRESS_EXTENSION 0x0
/begin IF_DATA CAN_EXT
120
LINK_MAP "AE0DAQ0O41" 0x80008116 0x0 0 0x2 1 0x2F 0x1
DISPLAY 0 0 655
/end IF_DATA
SYMBOL_LINK "AE0DAQ0O41" 0
/end MENT
How do we do that ??? because it has multiple lines like this ????
Thanks in advance!!!!!!!
If you consider your File 2 as a tab seperated value file then you could read the File 1 line by line and then compare the keyword in the file1 with each line in file2.
When you get a match then write another file with the new inputs
Quick and dirty solution:
(Assuming that the inputs are both text files...)
The code creates a dictionary by mining the second file.
The first file is processed line by line and written to the output file after the required modifications.
This is certainly not the best way to go about it.
If you know the exact format of the files, you can optimize the code to run a lot faster.
fout = open('output.txt' , 'w')
beg, ecu, lnk = '/begin','ECU_ADDRESS', 'LINK_MAP'
keyVal = dict()
with open('file2.txt') as f2:
for line in f2:
b = line.split(' ')
newK, newV = b[-1].replace('\n','') , b[-2].split('+')[0]
keyVal[newK] = newV
with open('file1.txt') as f1:
value,keyword = '', ''
for line in f1:
a = line.split(' ')
loc = 0
if beg in a and 'MENT' in a:
keyword = a[a.index(beg)+2]
value = '0x'+keyVal.get(keyword,keyword)
elif ecu in a:
loc = a.index(ecu) + 1
elif lnk in a: loc = a.index(lnk) + 2
else : loc = 0
if loc != 0:
a[loc] = value
a = ' '.join(a)
fout.writelines(a)
fout.close()

DB2 insert UTF-8 characters on non unicode database with ALT_COLLATE UNICODE

I am trying to insert Chinese text in a DB2 Database but not working.
The database is configured by default as ANSI (en_US 819) (and it's a requirement for other applications that use the dame databse) ALT_COLLATE IDENTITY_16BIT is defined and UNICODE tables are created using CCSID UNICODE but unicode characters for Chinese or Korean are not inserted.
Example table:
CREATE TABLE LANGS (
IDIOMA char(2) NOT NULL,
PAIS char(2) NOT NULL,
TRADUC long varchar NOT NULL,
) CCSID UNICODE;
Example insert:
INSERT INTO LANGS (IDIOMA,PAIS,TRADUC) VALUES ('zh','TW','其他');
System Information:
Server: DB2 9.7 on Ubuntu 64bit (en_US)
Client: Windows 7 32bit (es_ES) Java 7 with db2jcc.jar
Example Java extract:
Class.forName("com.ibm.db2.jcc.DB2Driver");
...
Properties props = new Properties();
props.setProperty("user", user);
props.setProperty("password", pass);
props.setProperty("DB2CODEPAGE", "1208");
props.setProperty("retrieveMessagesFromServerOnGetMessage", "true");
con = DriverManager.getConnection(url, props);
...
Statement statement = con.createStatement();
statement.execute(sql);
...
statement.close();
con.close();
DB cfg get
DB2 Database locale configuration
Territorio de base de datos = en_US;
Página de códigos de base de datos = 819
Conjunto de códigos de base de datos = iso8859-1
Código de país/región de base de datos = 1
Secuencia de clasificación de base de datos = UNIQUE
Orden de clasificación alternativo (ALT_COLLATE) = IDENTITY_16BIT
Tamaño de página de base de datos = 4096
Statements are executed correctly and rows appears correctly in the database for:
en_GB
en_US
es_ES
pt_PT
but not for:
cy_GB
ko_KR
zh_TW
Insert from command line with db2cmd also does not work for this languages (Inserts but with only 1 byte.
Insert from command line in a Linux environment localized as zh_TW works.
Insert from command line in a Linux environment localized as en_US.utf-8 works.
Never work on Java on these environments.
Using "X" as prefix form the VARCHAR field is not an option due some restrictions and the SQL works on two environments.
I think it may be some encoding problem on Client, or server due to configuration, file or sql encoding.
Update:
I tried also to load a UTF-8 file with the SQLs. the file loads correctly and debugging the SQL with UTF-8 characters is correctly passed to the Statement but the result is the same.
new InputStreamReader(new FileInputStream(file),"UTF-8")
...
private void executeLineByLine(Reader reader) throws SQLException {
StringBuffer command = new StringBuffer();
try {
BufferedReader lineReader = new BufferedReader(reader);
String line;
while ((line = lineReader.readLine()) != null) {
command = handleLine(command, line);
}
checkForMissingLineTerminator(command);
} catch (Exception e) {
String message = "Error executing: " + command + ". Cause: " + e;
printlnError(message);
throw new SQLException(message, e);
}
}
private StringBuffer handleLine(StringBuffer command, String line) throws SQLException, UnsupportedEncodingException {
String trimmedLine = line.trim();
if (lineIsComment(trimmedLine)) {
println(trimmedLine);
} else if (commandReadyToExecute(trimmedLine)) {
command.append(line.substring(0, line.lastIndexOf(delimiter)));
command.append(LINE_SEPARATOR);
println(command);
executeStatement(command.toString());
command.setLength(0);
} else if (trimmedLine.length() > 0) {
command.append(line);
command.append(LINE_SEPARATOR);
}
return command;
}
private void executeStatement(String command) throws SQLException, UnsupportedEncodingException {
boolean hasResults = false;
Statement statement = connection.createStatement();
hasResults = statement.execute(command);
printResults(statement, hasResults);
statement.close();
}
Update2:
It's not possible to change the data types. The database is part of other systems and already with data.
The database is installed on 7 different servers on three of it that the data is inserted using Linux in a UTF-8 shell the data was inserted correctly from db2 command line.
From windows db2 command line or using Java it's not possible to insert the characters correctly.
Changing the Java sources to UTF-8 source makes the System.out prints the SQL correctly like i see debugging the sql variable.
When i insert this test SQL. It is shown correctly with chines characters in the System.out and in the Statement internal variable
INSERT INTO LANGS (IDIOMA,PAIS,TRADUC) VALUES ('zh','TW','TEST1 其他 FIN TEST1');
But in the database the test appears as:
TEST3 FIN TEST3
HEX reprentation:
54 45 53 54 33 20 1A 1A 1A 1A 1A 1A 1A 1A 20 46 49 4E 20 54 45 53 54 33
T E S T 3 _ ? ? ? ? ? ? ? ? _ F I N _ T E S T 3
I think that probably DB2 Java client is using allways Windows codepage (in this case is ISO-8859-1 or cp1252) instead of UTF-8 or the server is converting the data using the main collate instead the alternative collate of the table.
Update3:
I installed a Java SQL tool called DbVisualizer and using this tool on windows when a paste in the SQL panel the SQL and run it is inserted correctly in the databse.
This makes me to suspect that is not a problem of installation or data types. Probably are one of this three factors.
Client configuration
Server properties sended when client connects
Driver type of version used
Problem is solved using these steps:
Use always db2jcc4.jar not db2jcc.jar (JDBC 4)
(In some places JDBC level 2 was configured in the OS classpath with db2jcc instead DB2jcc4 )
Set the environment variable DISABLEUNICODE=0
There is a complete information in this page Understanding DB2 Universal Database character conversion about unicode on DB2

Writing a .pdf to sd card in android

When I have a PDF formatted like so (abbreviated)
"JVBERi0xLjQKJf////8KMzIgMCBvYmoKPDwvTGVuZ3RoIDI0ODIKL1N1YnR5cGUgL1hNTAovVHlwZSAvTWV0YWRhdGEKPj4Kc3RyZWFtCjw/eHBhY2tldCBiZWdpbj0n77u/JyBpZD0nVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkJz8+Cjx4OnhtcG1ldGEgeDp4bXB0az0iMy4xLTcwMSIgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iPgogIDxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+CiAgICA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIiB4bWxuczp4bXA9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8iPgogICAgICA8eG1wOkNyZWF0ZURhdGU+MjAxMi0wOS0yNlQxNzoxOTo0OVo8L3htcDpDcmVhdGVEYXRlPgogICAgICA8eG1wOkNyZWF0b3JUb29sPk5pdHJvIFJlYWRlciAyICAoMi4gNS4gMC4gNDApPC94bXA6Q3JlYXRvclRvb2w+CiAgICAgIDx4bXA6TW9kaWZ5RGF0ZT4yMDEyLTA5LTI2VDE3OjE5OjQ5WjwveG1wOk1vZGlmeURhdGU+CiAgICAgIDx4bXA6TWV0YWRhdGFEYXRlPjIwMTItMDktMjZUMTc6MTk6NDlaPC94bXA6TWV0YWRhdGFEYXRlPgogICAgPC9yZGY6RGVzY3JpcHRpb24+CiAg
I am able to write it to the sd card with this code, and then open it.
File file = new File(eqnxPath.getAbsolutePath(), FileId + FileName);
Log.d(TAG, "looking for file...");
if (file.exists())
{
Log.d(TAG, "fiel exists, returning file");
return file;
}
else
{
Log.d(TAG, "file does not exist, making file");
byte[] pdfAsBytes = Base64.decode(this.FileContent, Base64.DEFAULT);
try
{
FileOutputStream os = new FileOutputStream(file);
try
{
os.write(pdfAsBytes);
os.flush();
os.close();
Log.d(TAG, "returning file");
return file;
} catch (IOException e)
{
Log.d(TAG, "IOException...");
e.printStackTrace();
}
} catch (FileNotFoundException e)
{
Log.d(TAG, "FileNotFoundException...");
e.printStackTrace();
}
However when I get a pdf that looks like this (again abbreviated)
%PDF-1.3
%����
1 0 obj
<undefined</Metadata 35 0 R/Pages 2 0 R/Type/Catalog>>
endobj
2 0 obj
<</MediaBox[0.0 0.0 609.12 788.88]/Count 1/Type/Pages/Kids[4 0 R]>>
endobj
4 0 obj
<undefined</Parent 2 0 R/Contents 29 0 R/PieceInfo<undefined</MRC<undefined</Private<undefined</B[1 1 3 1]/I[3 8 300 300]/P[2 0.45]/Q 0.1/S[2 0]/U 0/Y 15/b 20/v 2.31>>/LastModified(D:20121102151622-06'00')>>/PSL<undefined</Private<undefined</V(3.1.902)>>/LastModified(D:20121102211626-00'00')>>>>/MediaBox[0 0.0 609.12 788.88]/Resources<undefined</XObject<</4 28 0 R/5 27 0 R/6 26 0 R/7 23 0 R/8 22 0 R/9 21 0 R/A 20 0 R/B 19 0 R/C 18 0 R/D 17 0 R/E 16 0 R/F 15 0 R/G 14 0 R/H 13 0 R/I 12 0 R/J 11 0 R/K 10 0 R/L 9 0 R/M 8 0 R/N 7 0 R/O 6 0 R/P 5 0 R>>/ProcSet[/PDF/ImageC/ImageI/ImageB]>>/Type/Page/LastModified(D:20121102151622-06'00')>>
endobj
5 0 obj
<undefined</Subtype/Image/Length 20602/Filter/CCITTFaxDecode/ImageMask true/BitsPerComponent 1/Width 2320/DecodeParms<undefined</K -1/Columns 2320>>/Height 2960/Type/XObject>>stream
����KH�r�f(A�:FD���;!�(CǕ��:!������v�p���������������?w�A�?I<��
#� b��r�P�����������O��̓�Gcb��ɘ�("El%A�#���r��S����3����h� �� �Ȑc"#�.�0DE��2PA�.(A�a�8k�)%���L��#A�c'd�Ȑh*f� `�M �(�f�C6���� ��xN��� �0 xA��� `�A�h�2�&�y�TA�x'A��zp����4L a��� 駬?OO�S�������xA���~��N,'��AƐN-;���i����[�[OD~�Zo��q�"2-��*v��[���m�q�!֓rM�'��g8rC�1�`�����%�G� :!r��"�4fB� G��h�)2c&�tpO��O ��W��GSAA���:�O!�ɧ�`��5��n#�'���zM�ap���_�&�I�J �n��L8OA�����&�=W ��xN���H=7 �=7
�!xV��n����n�]7���I�n��.�����&�[���O[���z�Kn~���_�������I�����V��������Z�������������������������"A����������
���ܫ��������������e��� a������/����H3?���������_�ߐq������������ �7�������O�����/��/�!�E���������"�u���������� D���� ���肋��������� ������� ������U����D����Ai��������������_�����\6i������ݿ�K����V�/� c������_���m���0K���"�zo��������c���������������_�������������_���]��������k����_�~�����������������������D������������w��k�������]���������o�J������E����/ۯ����
�_�Z�BA/��^�Jյ���KJ��/���k��[�յm+Xui��������i{a[K�:lڤ�IZ���Z�����k%
v�
��^�0}���$}��cRCRCH�4�l+iCY
m/�a~
)6� L�� ޖ%�lW��3��ȑ�+ �m)�Hb�a/�?��b�67د
5�?�*����**(1_��5��^�[�2
/����5U_쁈�S[�
.�^ �����P��/��A���
B
n������/�va�Bk
���/�
E�����a4"9���"+�
0� �"�&���Xk�aa��`�!� ���2�=��j� ��A��� ����l��e�A������SǨ�����/�_�������:���������
���a!VSZ��������!*�H7龓��W��̓�
��"Ő1��)%
#�;D��)9!� #AAld �`�>�P�u�"D�
9|�rundefined<azv�z�0����zi����za;���L'�קj�A��OT�7^-�'���4�M��\�� G���v�c�<�P��O �o���I�t��M� ������ �zN�y�0�>�:]=����2�5�:O�Ӿ��ivAA?[�ut���S��7a�^��׽�^���P�N����n��Upa�_X�����
���_ޗ�<����Zo��o�[��U�e�X�����_��7��D�]m������#ٷ��_\��������������/u����������������SP+��������#ڧ���������������{��_����^��]�;���^����h�������W����������������د�n����������i[_��]�ڵ�[]�
����K��J�J����m-���յ��]�xk��J�X/�����1V����0�����V!�����_�h5
o�����P�/��O����P����_R�13��RBL���G"Cn�l���K���d���}kV������eP�����Z��� _�e�*Bl� 0ed�3;Z`��9i*��5%����d�iw�z���u����$�~4�߅��o�Q���������ܧ2H.x�X�џ)��Ρ�p�8S�4g'� 0hGS'�<� �=B`�;��u=< �4�?M4� �S��^�ń���M>/�N���ӽPh�v��� ��#�t�܉�Dt!9���O��"��#�����=$��龃A�n�#�7M�o����I=>��t��U:N����T���}7��[WT�+������ .�_�C�=>��=p����������6c6zVf�ޥ�mB� ��?����D^�j��G������������?������-�_������W������$�$c��L� �ד�����FGJFG���鷷��o���u���u��u_^�/����_��}-���ki}�w�u.���Z�v�����Z�]�~��mv�V��Nڶ��k�Z���I��&����L5`�]�}�`˃�lSTW��LS�HM�����
��ڦ�� �i����a���
a��k�����4���+a�A�^��ӴװX0D�&A�M!iq��D�"#������Sz�~�_j�#���������������s\�
٪3��B��)_Q���-Of��^��������?���������_�������_�%����̔����}��EW���TY�~.�5�q�������������������l����:���#��ddj��\φ%�(RFvjɺ� �Oe;j��Yrundefined<W4���j�ddP3S7�`�DNN!K�����0��H!���j�i����!�:3XR��Ä �0���Au�M���M=>&��MQ>!H�\�At�O��8w
{��Z�
$�%N����7�����6r�7�����u��ץ�5�o#��y tO�6 M6��s1�� ���D��v�".9�k&�O*'�d�e�_�0��t�J��U����Fe'ۗ6�ӧk߄����q�+��i����.�w��_�xA��:�}��&��q��oҽ/�Y��!�ҿ_�?�_��1�����M�?�/�ӏ}�_~�_���T����==iid�������W�׭��7�WO�������������_�o����:�����R�_���!����Azվ���z�ȉ�W���Z�]b����W����o���-k�d?�_������~���M����p�_�$��uMR�&-��)����.��o�.�Ip��������$�����������k_��/y/�W���#�~��M�+�,�7��^��5�� �"_ҧ�����qqZ�ﭨ5#��g�{��|A�Σ���J�/�I/��zm'�un�}��������II�)�����_�t�Hw��}���%_�W���K�'�������
�u|^���'�+��/�v�ߧ聟���9�~�Z���w��Z�/�K�\?��^����}-t����ץ��u���v6֟��������]h/^�ƕ{���]��^�R��mwUV�R�Y�%�yѯ�����ۮ�Z���N����}p�5�]+K����U�����y�vҤ
*�����&��I1\���:!���f�0�|V�/��I5���[ˋ�`��SH��a%�b!�
�Ay����aWZU�`�rG�����A��+��E!q��
/����-��X`�0��\�V��E'���[�_��I?���-yᅿ�_�0[�L,4�Imt��zB"8��������0�
;P���)�,��lE�f4�qZ��^��MUZhDb?����j|��"�\���TY*jFǟ�"!D�b���!.�Fdv��*�NdX!6(�����
0����Y�A���gSDԇ��Û�V�A�x �ja��=5��T���C�����ih4�a=4Ӫ����D��|��[�Ӈpk�޿wڧ��:Dp��O(��¥c׾�?֗OY(|�Q�r;�` �������t6��M���K0�����'ϒƲ��'��"%ԏ��(+��n�+������u���
��A�e֛�oKH_��۽������OW��������[_���RYC��޵kh����I�o&�G�ѪZ����w~�-��?���U/���������i/��W߮��[_�����������|$��.��խ������JA6������^D �ҥ����A��J��*���O���5���~�������o�[�T����m/��_����]r �����R���I%����/��lz/M�����0�����W���\��Pa���O��^G��6��%�����z��a�u���U�D0�9��
���Ҧ�K�p�έ����[����Z��?�3������߂��<��XJ�����Km/�����:�#��I~�R�I�]���^���#������ץ���/����ޕ����A'^�.�/�k~��Z �k��դ��-źJ���~�v����k��AWK�������֯�_��^��U�0�Natˉ��%�ܘ�3�V��0_����y��xk��!���d��)#�undefined<BJ-��1QE��(����A��`�QR�H���jCG
4E�i�װ�
}K1(>�1\;�*��������8[�׿k
��n�[ }�%�
�h��7]���Xk�i�ͅ�+�DG�CB
�pf�����E��d*c��4z�I*���J��P�!��
My method does not work. How would I write this type of .pdf to the sd card? I dont even know what type of data I am looking at...
A PDF "formatted" like in your first example is base64 encoded (an encoding used to transport binary data via a channel that can only handle text properly).
Before writing to card, therefore, it has to be decoded which your method does by means of a ' Base64.decode' call.
A PDF "formatted" like in your second example is already in its unencoded state. Thus, no further decoding is necessary and the data has to be written to card as is.
I hope your member variable ' this.FileContent' is not a string but a byte array, or that its byte array representation is present in yet another member. Otherwise the second, unencoded form surely already is broken.
For this kind of data your method shall not call the base64 decode method before saving but instead save the data as is.

Search & display results [java]

I run a small online gaming community and deal with a database of accounts.
The setup is this:
Folder named Accounts
Inside the Accounts directory, there is 200,000+ text files organized by player name. Access to this folder manually is a pain because of the needed RAM to get in and search files. I find this very inconvenient.
I access this directory to send password reminders or for highscores on who has been playing the longest.
Here is an example of an account file. This file is named Falcon.txt
[ACCOUNT]
character-username = Falcon
character-password = falconpassword
[INFO]
character-coordx = 3252
character-coordy = 3432
character-active = yes
character-ismember = 1
character-messages = 5
character-lastconnection = [removed]
character-lastlogin = 2009-11-29
character-energy = 100
character-gametime = 193
character-gamecount = 183
[EQUIPMENT]
character-equip = 0 4724 0
character-equip = 1 1052 0
character-equip = 2 6585 0
character-equip = 3 4151 0
character-equip = 4 4720 0
character-equip = 5 1215 0
character-equip = 6 -1 0
character-equip = 7 4722 0
character-equip = 8 -1 0
character-equip = 9 775 0
character-equip = 10 1837 0
character-equip = 11 -1 0
character-equip = 12 6735 0
character-equip = 13 -1 0
[APPEARANCE]
character-look = 0 1
character-look = 1 1
character-look = 2 2
character-look = 3 3
character-look = 4 5
character-look = 5 2
[STATS]
character-skill = 0 1 0
character-skill = 1 1 0
character-skill = 2 1 0
character-skill = 3 1 0
character-skill = 4 1 0
character-skill = 5 1 0
character-skill = 6 1 0
character-skill = 7 1 0
character-skill = 8 1 0
character-skill = 9 1 0
character-skill = 10 1 0
character-skill = 11 1 0
character-skill = 12 1 0
character-skill = 13 1 0
character-skill = 14 1 0
character-skill = 15 1 0
character-skill = 16 1 0
character-skill = 17 1 0
character-skill = 18 1 0
character-skill = 19 1 0
character-skill = 20 1 0
[ITEMS]
[BANK]
[FRIENDS]
[IGNORES]
[END]
There is a huge database of these and search through the directory in the files for values.
Values I mean by item ID's or IP addresses to find and track other accounts.
However I have a new problem and my development for this is crashing.
As you can see in the file the lines are organized by tabs.
character-equip = 0 4724 1
If I put the value 4724 in my search application, I want it to print out the value 1 tab to the right of the found search result. I want it to print out the value for the found results only, not extra results.
So the search could look like this:
1 "Enter item to find:"
2 "Enter item to find: 4724"
3 "Account Falcon.txt has 1!"
press any key to continue...
Or if there was more quantity of that equipped item
character-equip = 5 1239 102
1. "Enter item to find:"
2. "Enter item to find: 1239"
3. "Account Falcon2.txt has 102!"
press any key to continue...
I simply want to input an item ID, and have it display the value after the found value. The white space is a tab. I have tried doing this and the only successful way of getting any result is to put a tab in between the search term. So if I want to find item 1239 id type this in the cmd line:
Enter item to find:<tab>1239<tab>
It would then search for that and will display the accounts with that item in it. However I still have to individually open up the accounts to find out the quantity of that item. I want the search results to display the quantity of the item if the value is found. However if the value is a quantity and it trys to search one tab over, I want it to either skip it or say zero.
This is what I mean.
character-equip = 0 1024 1239
Enter item to find: 1239
If it hits this account I want to make the search results display a zero if it cannot tab over or view any values in the white space. So it will display as null or zero
Account Falcon3.txt has null!
or
Account Falcon3.txt has 0!
I've attempted to do this but I am unsure how to achieve this.
Here is my code.
import java.io.*;
import java.util.*;
public class ItemDatabase {
public static void main(String args[]) {
System.out.print("Enter item to find: ");
Scanner sc = new Scanner(System.in);
find(sc.nextLine());
}
public static void find(String delim) {
File dir = new File("accounts");
if (dir.exists()) {
String read;
try {
File files[] = dir.listFiles();
for (int i = 0; i < files.length; i++) {
File loaded = files[i];
if (loaded.getName().endsWith(".txt")) {
BufferedReader in = new BufferedReader(new FileReader(loaded));
StringBuffer load = new StringBuffer();
while ((read = in.readLine()) != null) {
load.append(read + "\n");
}
String delimiter[] = new String(load).split(delim);
if(delimiter.length > 1) {
System.out.println("Account " + loaded.getName() + "has " + delimiter[1] + "!");
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
} else {
System.out.println("error: dir wasn't found!");
}
}
}
Thanks guys I hope you can help me.
This is simply crying out for a database. If your entire back end is running in a single java process I'd recommend going with something like Apache Derby or H2.
If you'd rather not move to a database, and the main problem is the act of listing all the entries in a single directory, could you split it into a heirarchy. Your Falcon account could then be located in F/FA/Falcon.txt - each directory would then contain a more manageable number of files.
Aside from the need for a database, you could probably implement your solution more intuitively and easily using commandline utilities such as find, grep, etc. or a text-processing language such as perl.
Maybe
grep '[0-9]+\t[0-9]+\t1239' Account_Falcon3.txt
Would return
character-equip = 0 1024 1239
You could then easily see that the value is 0 for that item.
I cannot emphasize enough the need to not write a Java program to do this - you won't do as good a job as the authors of the standard shell utilities. Let's face it, the fact that you are asking this question indicates that you are a newb! :) (We are all newbs depending on the topic).

Categories