We Keep Coding

Issue in parsing TLV data in java, how to get value,length

I have tried many sample codes to parse APDU response to TLV format.
I am able to parse it properly if the response length is less but facing issue if length is more(how calculate length of a tag without any libraries)
NOTE: I am using predefined tags in Constants
code:
private HashMap<String, String> parseTLV(String apduResponse) {
HashMap<String, String> tagValue = new HashMap<>();
String remainingApdu = apduResponse.replaceAll(" ", "");
if (remainingApdu.endsWith(ResponseTags._SUCCESS_STATUSWORDS)) {
remainingApdu = remainingApdu.substring(0, remainingApdu.length() - 4);
}
while (remainingApdu != null && remainingApdu.length() > 2) {
remainingApdu = addTagValue(tagValue, remainingApdu);
}
return tagValue;
}
addTagValue method
private String addTagValue(HashMap<String, String> tagValue, String apduResponse) {
String tag = "";
String length = "";
String value = "";
int tagLen = 0;
if (tagUtils.isValidTag(apduResponse.substring(0, 2))) {
tagLen = readTagLength(apduResponse.substring(3));
// tagLen = 2;
tag = apduResponse.substring(0, 2);
} else if (tagUtils.isValidTag(apduResponse.substring(0, 4))) {
tagLen = 4;
tag = apduResponse.substring(0, 4);
} else {
return "";
}
Log.e("TAG_LEN","tag: "+tag+"taglen: "+tagLen);
if (tagUtils.shouldCheckValueFor(tag)) {
length = apduResponse.substring(tagLen, tagLen + 2);
int len = tagUtils.hexToDecimal(length);
value = apduResponse.substring(tagLen + 2, (len * 2) + tagLen + 2);
tagValue.put(tag, value);
if (ResponseTags.getRespTagsmap().containsKey(tag)) {
//logData = logData + "\nKEY:" + tag + " TAG:" + ResponseTags.getRespTagsmap().get(tag)/* + " VALUE:" + value + "\n "*/;
}
if (tagUtils.isTemplateTag(tag)) {
// logData = logData + "\n\t-->";
return addTagValue(tagValue, value) + apduResponse.substring(tag.length() + value.length() + length.length());
} else {
return apduResponse.substring(tag.length() + value.length() + length.length());
}
} else {
value = apduResponse.substring(2, 4);
tagValue.put(tag, value);
// logData = logData + "\n\t\tKEY:" + tag + " TAG:" + ResponseTags.getRespTagsmap().get(tag) /*+ " VALUE:" + value + "\n "*/;
return apduResponse.substring(tag.length() + value.length() + length.length());
}
}
readTagLength :
private int readTagLength(String apduResponse) {
int len_bytes = 0;
if (apduResponse.length() > 2) {
len_bytes = (apduResponse.length()) / 2;
}
Log.e("tlv length:", "bytes:" + len_bytes);
if (len_bytes < 128) {
return 2;
} else if (len_bytes > 127 && len_bytes < 255) {
return 4;
} else {
return 6;
}
}
I cannot able to get length properly for few cards(if apdu response is long)
Please help

First be sure the input data is proper before you go into the code. Take the full data and try it on https://www.emvlab.org/tlvutils/ .
Once its confirmed the data is proper, go through in EMV 4.3 Book 3,
Annex B Rules for BER-TLV Data Objects sections B1, B2, B3 - with utmost attention.
If you follow this precisely, then you wouldn't need to store a static list of tags; will save time in future.

Below sample has an assumption that TLV array is ending with special 0x00 tag but for sure you can ignore it.
Pojo class :
public class Tlv {
private short tag;
private byte[] value;
public Tlv(short tag) {
this.tag = tag;
}
public short getTag() {
return tag;
}
public byte[] getValue() {
return value;
}
public void setValue(byte[] valueBytes) {
this.value = valueBytes;
}
}
Utility method :
public static Map<Byte, Tlv> parse(ByteBuffer bb) throws TlvException {
Map<Byte, Tlv> tlvs = null;
tlvs = new HashMap<Byte, Tlv>();
try {
while (bb.remaining() > 0) {
byte tag = bb.get();
if(tag == 0x00)
continue;
int length = bb.get();
byte[] value = new byte[length];
bb.get(value, 0, length);
Tlv tlv = new Tlv(tag);
tlv.setValue(value);
tlvs.put(tag, tlv);
}
} catch (IndexOutOfBoundsException e) {
throw new TlvException("Malformed TLV part: " + bb.toString() + ".", e);
}
return tlvs;
}

Http range Streaming video file too html5 video while downloading via BitTorrent protocol Spring MVC

I am developing a "Torrent Streamer" website for a school project.
We use:
Tomcat
Spring boot starter ( MVC )
Java 1.8 (can't upgrade)
Jlibtorrent for the torrent management.
Xuggle-video 1.3.6 for video conversion.
We have to:
1 = download a torrent with Bit torrent protocol. (done)
2 = convert video file or part of video file to mp4. (done)
3 = start to stream it even if download is not complete. (fail)
I tried to give the getVideoRange as source of the "video". With getVideoRange, the web browser loop until EOFexception. And i can see the video if the file is already downloaded. Don't work at all while downloading.
With the multipartfile way I can get the video, but not entirely, and it's buggy if the file is not entirely downloaded. (From https://github.com/davinkevin/Podcast-Server.)
I would like to understand what I am missing in the HTTP range protocol and/or in the way I have to use SPRING to stream a video file myself to a HTML5 video player.
Is there a way to do it?
Controller.
#GetMapping("getVideoRange")
public ResponseEntity<ByteArrayResource> getVideoRange(HttpServletRequest request, HttpServletResponse response) throws IOException{
boolean finished = false;
RandomAccessFile file = new RandomAccessFile("torrents/Zenith.Part.1.2011.Theora-VODO/Zenith.Part.1.2011.Theora-VODO.ogv" , "r");
long filelen = file.length();
Path filepath = Paths.get("torrents/Zenith.Part.1.2011.Theora-VODO/Zenith.Part.1.2011.Theora-VODO.ogv");
String rangestring= request.getHeader("Range");
//System.out.println(rangestring);
String[] strings = (rangestring.substring(6)).split("-");
//System.out.println("start = " + strings[0]);
//System.out.println("filelen = "+filelen );
int start = Integer.parseInt(strings[0]);
int len = 64*8*1024;
if(filelen - start < len)
{
len = (int) (filelen-start) ;
}
byte[] b = new byte[len];
//System.out.println("Range asked = " + (filelen -start));
//System.out.println("read len = " + len + "\n");
try {
file.seek(start);
file.readFully(b, 0, len);
file.close();
} catch (EOFException e) {
finished = true;
} catch (IOException e) {
System.out.println(e);
System.out.println(e.getMessage());
e.printStackTrace();
return new ResponseEntity<ByteArrayResource>(HttpStatus.INTERNAL_SERVER_ERROR);
}
ByteArrayResource bResource = new ByteArrayResource(b);
b = null;
String fileName = filepath.getFileName().toString();
String contentType = "video/ogg";
HttpHeaders responseHeader = new HttpHeaders();
responseHeader.add("Content-Type", contentType);
responseHeader.add("ETag", fileName);
responseHeader.add("Content-Length", ""+(len) +"");
responseHeader.add("Accept-Ranges", "bytes");
responseHeader.add("Content-Range", "bytes "+ start +"-"+(start + len - 1)+"/"+filelen);
return new ResponseEntity<ByteArrayResource>(bResource, responseHeader ,HttpStatus.PARTIAL_CONTENT);
}
#GetMapping("multipartfile")
public void getEpisodeFile( HttpServletRequest request, HttpServletResponse response) throws Exception {
Path path = Paths.get("torrents/Zenith.Part.1.2011.Theora-VODO/Zenith.Part.1.2011.Theora-VODO.ogv");
try {
MultipartFileSender.fromPath(path).with(request).with(response).serveResource();
} catch (Exception e) {System.out.println(e);// TODO: handle exception
}
}
MultipartFileSender.
package com.asv.hypertube.torrent;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.*;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.FileTime;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.ZoneOffset;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
/**
* Created by kevin on 10/02/15.
* See full code here : https://github.com/davinkevin/Podcast-Server/blob/d927d9b8cb9ea1268af74316cd20b7192ca92da7/src/main/java/lan/dk/podcastserver/utils/multipart/MultipartFileSender.java
*/
public class MultipartFileSender {
protected final Logger logger = LoggerFactory.getLogger(this.getClass());
private static final int DEFAULT_BUFFER_SIZE = 20480; // ..bytes = 20KB.
private static final long DEFAULT_EXPIRE_TIME = 604800000L; // ..ms = 1 week.
private static final String MULTIPART_BOUNDARY = "MULTIPART_BYTERANGES";
Path filepath;
HttpServletRequest request;
HttpServletResponse response;
public MultipartFileSender() {
}
public static MultipartFileSender fromPath(Path path) {
return new MultipartFileSender().setFilepath(path);
}
public static MultipartFileSender fromFile(File file) {
return new MultipartFileSender().setFilepath(file.toPath());
}
public static MultipartFileSender fromURIString(String uri) {
return new MultipartFileSender().setFilepath(Paths.get(uri));
}
//** internal setter **//
private MultipartFileSender setFilepath(Path filepath) {
this.filepath = filepath;
return this;
}
public MultipartFileSender with(HttpServletRequest httpRequest) {
request = httpRequest;
return this;
}
public MultipartFileSender with(HttpServletResponse httpResponse) {
response = httpResponse;
return this;
}
public void serveResource() throws Exception {
if (response == null || request == null) {
return;
}
if (!Files.exists(filepath)) {
logger.error("File doesn't exist at URI : {}", filepath.toAbsolutePath().toString());
response.sendError(HttpServletResponse.SC_NOT_FOUND);
return;
}
Long length = Files.size(filepath);
String fileName = filepath.getFileName().toString();
FileTime lastModifiedObj = Files.getLastModifiedTime(filepath);
if (StringUtils.isEmpty(fileName) || lastModifiedObj == null) {
response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
return;
}
long lastModified = LocalDateTime.ofInstant(lastModifiedObj.toInstant(), ZoneId.of(ZoneOffset.systemDefault().getId())).toEpochSecond(ZoneOffset.UTC);
String contentType = "video/mp4";
// Validate request headers for caching ---------------------------------------------------
// If-None-Match header should contain "*" or ETag. If so, then return 304.
String ifNoneMatch = request.getHeader("If-None-Match");
if (ifNoneMatch != null && HttpUtils.matches(ifNoneMatch, fileName)) {
response.setHeader("ETag", fileName); // Required in 304.
response.sendError(HttpServletResponse.SC_NOT_MODIFIED);
return;
}
// If-Modified-Since header should be greater than LastModified. If so, then return 304.
// This header is ignored if any If-None-Match header is specified.
long ifModifiedSince = request.getDateHeader("If-Modified-Since");
if (ifNoneMatch == null && ifModifiedSince != -1 && ifModifiedSince + 1000 > lastModified) {
response.setHeader("ETag", fileName); // Required in 304.
response.sendError(HttpServletResponse.SC_NOT_MODIFIED);
return;
}
// Validate request headers for resume ----------------------------------------------------
// If-Match header should contain "*" or ETag. If not, then return 412.
String ifMatch = request.getHeader("If-Match");
if (ifMatch != null && !HttpUtils.matches(ifMatch, fileName)) {
response.sendError(HttpServletResponse.SC_PRECONDITION_FAILED);
return;
}
// If-Unmodified-Since header should be greater than LastModified. If not, then return 412.
long ifUnmodifiedSince = request.getDateHeader("If-Unmodified-Since");
if (ifUnmodifiedSince != -1 && ifUnmodifiedSince + 1000 <= lastModified) {
response.sendError(HttpServletResponse.SC_PRECONDITION_FAILED);
return;
}
// Validate and process range -------------------------------------------------------------
// Prepare some variables. The full Range represents the complete file.
Range full = new Range(0, length - 1, length);
List<Range> ranges = new ArrayList<>();
// Validate and process Range and If-Range headers.
String range = request.getHeader("Range");
if (range != null) {
// Range header should match format "bytes=n-n,n-n,n-n...". If not, then return 416.
if (!range.matches("^bytes=\\d*-\\d*(,\\d*-\\d*)*$")) {
response.setHeader("Content-Range", "bytes */" + length); // Required in 416.
response.sendError(HttpServletResponse.SC_REQUESTED_RANGE_NOT_SATISFIABLE);
return;
}
String ifRange = request.getHeader("If-Range");
if (ifRange != null && !ifRange.equals(fileName)) {
try {
long ifRangeTime = request.getDateHeader("If-Range"); // Throws IAE if invalid.
if (ifRangeTime != -1) {
ranges.add(full);
}
} catch (IllegalArgumentException ignore) {
ranges.add(full);
}
}
// If any valid If-Range header, then process each part of byte range.
if (ranges.isEmpty()) {
for (String part : range.substring(6).split(",")) {
// Assuming a file with length of 100, the following examples returns bytes at:
// 50-80 (50 to 80), 40- (40 to length=100), -20 (length-20=80 to length=100).
long start = Range.sublong(part, 0, part.indexOf("-"));
long end = Range.sublong(part, part.indexOf("-") + 1, part.length());
if (start == -1) {
start = length - end;
end = length - 1;
} else if (end == -1 || end > length - 1) {
end = length - 1;
}
// Check if Range is syntactically valid. If not, then return 416.
if (start > end) {
response.setHeader("Content-Range", "bytes */" + length); // Required in 416.
response.sendError(HttpServletResponse.SC_REQUESTED_RANGE_NOT_SATISFIABLE);
return;
}
// Add range.
ranges.add(new Range(start, end, length));
}
}
}
// Prepare and initialize response --------------------------------------------------------
// Get content type by file name and set content disposition.
String disposition = "inline";
// If content type is unknown, then set the default value.
// For all content types, see: http://www.w3schools.com/media/media_mimeref.asp
// To add new content types, add new mime-mapping entry in web.xml.
if (contentType == null) {
contentType = "application/octet-stream";
} else if (!contentType.startsWith("image")) {
// Else, expect for images, determine content disposition. If content type is supported by
// the browser, then set to inline, else attachment which will pop a 'save as' dialogue.
String accept = request.getHeader("Accept");
disposition = accept != null && HttpUtils.accepts(accept, contentType) ? "inline" : "attachment";
}
logger.debug("Content-Type : {}", contentType);
// Initialize response.
response.reset();
response.setBufferSize(DEFAULT_BUFFER_SIZE);
response.setContentType(contentType);
response.addHeader("Content-Type", contentType);
response.setHeader("Content-Type", contentType);
response.setHeader("Content-Disposition", disposition + ";filename=\"" + fileName + "\"");
logger.debug("Content-Disposition : {}", disposition);
response.setHeader("Accept-Ranges", "bytes");
response.setHeader("ETag", fileName);
response.setDateHeader("Last-Modified", lastModified);
response.setDateHeader("Expires", System.currentTimeMillis() + DEFAULT_EXPIRE_TIME);
// Send requested file (part(s)) to client ------------------------------------------------
// Prepare streams.
try (InputStream input = new BufferedInputStream(Files.newInputStream(filepath));
OutputStream output = response.getOutputStream()) {
if (ranges.isEmpty() || ranges.get(0) == full) {
// Return full file.
logger.info("Return full file");
response.setContentType(contentType);
response.setHeader("Content-Range", "bytes " + full.start + "-" + full.end + "/" + full.total);
response.setHeader("Content-Length", String.valueOf(full.length));
Range.copy(input, output, length, full.start, full.length);
} else if (ranges.size() == 1) {
// Return single part of file.
Range r = ranges.get(0);
logger.info("Return 1 part of file : from ({}) to ({})", r.start, r.end);
response.setContentType(contentType);
response.setHeader("Content-Range", "bytes " + r.start + "-" + r.end + "/" + r.total);
response.setHeader("Content-Length", String.valueOf(r.length));
response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT); // 206.
// Copy single part range.
Range.copy(input, output, length, r.start, r.length);
} else {
// Return multiple parts of file.
response.setContentType("multipart/byteranges; boundary=" + MULTIPART_BOUNDARY);
response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT); // 206.
// Cast back to ServletOutputStream to get the easy println methods.
ServletOutputStream sos = (ServletOutputStream) output;
// Copy multi part range.
for (Range r : ranges) {
logger.info("Return multi part of file : from ({}) to ({})", r.start, r.end);
// Add multipart boundary and header fields for every range.
sos.println();
sos.println("--" + MULTIPART_BOUNDARY);
sos.println("Content-Type: " + contentType);
sos.println("Content-Range: bytes " + r.start + "-" + r.end + "/" + r.total);
// Copy single part range of multi part range.
Range.copy(input, output, length, r.start, r.length);
}
// End with multipart boundary.
sos.println();
sos.println("--" + MULTIPART_BOUNDARY + "--");
}
}
}
private static class Range {
long start;
long end;
long length;
long total;
/**
* Construct a byte range.
* #param start Start of the byte range.
* #param end End of the byte range.
* #param total Total length of the byte source.
*/
public Range(long start, long end, long total) {
this.start = start;
this.end = end;
this.length = end - start + 1;
this.total = total;
}
public static long sublong(String value, int beginIndex, int endIndex) {
String substring = value.substring(beginIndex, endIndex);
return (substring.length() > 0) ? Long.parseLong(substring) : -1;
}
private static void copy(InputStream input, OutputStream output, long inputSize, long start, long length) throws IOException {
byte[] buffer = new byte[DEFAULT_BUFFER_SIZE];
int read;
if (inputSize == length) {
// Write full range.
while ((read = input.read(buffer)) > 0) {
output.write(buffer, 0, read);
output.flush();
}
} else {
input.skip(start);
long toRead = length;
while ((read = input.read(buffer)) > 0) {
if ((toRead -= read) > 0) {
output.write(buffer, 0, read);
output.flush();
} else {
output.write(buffer, 0, (int) toRead + read);
output.flush();
break;
}
}
}
}
}
private static class HttpUtils {
/**
* Returns true if the given accept header accepts the given value.
* #param acceptHeader The accept header.
* #param toAccept The value to be accepted.
* #return True if the given accept header accepts the given value.
*/
public static boolean accepts(String acceptHeader, String toAccept) {
String[] acceptValues = acceptHeader.split("\\s*(,|;)\\s*");
Arrays.sort(acceptValues);
return Arrays.binarySearch(acceptValues, toAccept) > -1
|| Arrays.binarySearch(acceptValues, toAccept.replaceAll("/.*$", "/*")) > -1
|| Arrays.binarySearch(acceptValues, "*/*") > -1;
}
/**
* Returns true if the given match header matches the given value.
* #param matchHeader The match header.
* #param toMatch The value to be matched.
* #return True if the given match header matches the given value.
*/
public static boolean matches(String matchHeader, String toMatch) {
String[] matchValues = matchHeader.split("\\s*,\\s*");
Arrays.sort(matchValues);
return Arrays.binarySearch(matchValues, toMatch) > -1
|| Arrays.binarySearch(matchValues, "*") > -1;
}
}
}

Start reading an Excel sheet at a specified row

I am reading an Excel file using POI's XSSF and SAX (Event API). The Excel sheet has thousands of rows so this is the only way that I have found to have good performance. Now I would like to read Excel file from one particularly row, for example row 6 because these Excel files are updated every day and I store the last row that I have already stored so I can read only the new rows.
How can I start from one row?
private void getExcelField(AcquisitionForm acquisitionForm) throws ExcelReadException, IOException{
InputStream stream=null;
OPCPackage p=null;
try{
p = OPCPackage.open(acquisitionForm.getDatatablePath(), PackageAccess.READ);
ReadOnlySharedStringsTable strings = new ReadOnlySharedStringsTable(p);
XSSFReader xssfReader = new XSSFReader(p);
StylesTable styles = xssfReader.getStylesTable();
XSSFReader.SheetIterator iter = (XSSFReader.SheetIterator) xssfReader.getSheetsData();
int index = 0;
//Test with one sheet
iter.hasNext();
//(iter.hasNext()) {
stream = iter.next();
String sheetName = iter.getSheetName();
processSheet(styles, strings, stream, acquisitionForm);
stream.close();
//++index;
//}
p.close();
}catch(Exception e){
throw new ExcelReadException("An error occured during excel file reading ", e);
}finally{
if (stream!=null)
stream.close();
if (p!=null)
p.close();
//++index;
// }
}
}
/**
* Parses and shows the content of one sheet
* using the specified styles and shared-strings tables.
* #param styles
* #param strings
* #param sheetInputStream
* #throws ExcelReadException
*/
private void processSheet(StylesTable styles, ReadOnlySharedStringsTable strings, InputStream sheetInputStream, AcquisitionForm acquisitionForm) throws Exception {
InputSource sheetSource = new InputSource(sheetInputStream);
SAXParserFactory saxFactory = SAXParserFactory.newInstance();
SAXParser saxParser = saxFactory.newSAXParser();
XMLReader sheetParser = saxParser.getXMLReader();
//ContentHandler handler = new MyXSSFSheetHandler(styles, strings);
MyXSSFSheetHandler handler = new MyXSSFSheetHandler(styles, strings,databaseAcquisitionServices, acquisitionForm);
sheetParser.setContentHandler(handler);
sheetParser.parse(sheetSource);
}
MyXSSFSheetHandler
public MyXSSFSheetHandler(StylesTable styles, ReadOnlySharedStringsTable strings,DatabaseAcquisitionServices databaseAcquisitionServices, AcquisitionForm acquisitionForm, int sheetIndex) {
this.stylesTable = styles;
this.sharedStringsTable = strings;
this.formatter = new DataFormatter();
this.value = new StringBuffer();
this.nextDataType = XssfDataType.NUMBER;
this.databaseAcquisitionServices=databaseAcquisitionServices;
this.acquisitionForm=acquisitionForm;
this.sheetIndex = sheetIndex;
}
/**
*
*/
public void startElement(String uri, String localName, String name,
Attributes attributes) throws SAXException {
if ("inlineStr".equals(name) || "v".equals(name)) {
vIsOpen = true;
// Clear contents cache
value.setLength(0);
}
// c => cell
else if ("c".equals(name)) {
// Get the cell reference
cellCoordinate = attributes.getValue("r");
int firstDigit = -1;
for (int c = 0; c < cellCoordinate.length(); ++c) {
if (Character.isDigit(cellCoordinate.charAt(c))) {
firstDigit = c;
break;
}
}
thisColumn = nameToColumn(cellCoordinate.substring(0, firstDigit));
// Set up defaults.
this.nextDataType = XssfDataType.NUMBER;
this.formatIndex = -1;
this.formatString = null;
String cellType = attributes.getValue("t");
String cellStyleStr = attributes.getValue("s");
if ("b".equals(cellType)) {
nextDataType = XssfDataType.BOOL;
} else if ("e".equals(cellType)) {
nextDataType = XssfDataType.ERROR;
} else if ("inlineStr".equals(cellType)) {
nextDataType = XssfDataType.INLINESTR;
} else if ("s".equals(cellType)) {
nextDataType = XssfDataType.SSTINDEX;
} else if ("str".equals(cellType)) {
nextDataType = XssfDataType.FORMULA;
} else if (cellStyleStr != null) {
// It's a number, but almost certainly one
// with a special style or format
int styleIndex = Integer.parseInt(cellStyleStr);
XSSFCellStyle style = stylesTable.getStyleAt(styleIndex);
this.formatIndex = style.getDataFormat();
this.formatString = style.getDataFormatString();
if (this.formatString == null) {
this.formatString = BuiltinFormats.getBuiltinFormat(this.formatIndex);
}
}
}
}
/*
* (non-Javadoc)
* #see org.xml.sax.helpers.DefaultHandler#endElement(java.lang.String, java.lang.String, java.lang.String)
*/
public void endElement(String uri, String localName, String name)
throws SAXException {
String cellValue = null;
//String thisStr = null;
// v => contents of a cell
if ("v".equals(name)) {
// Process the value contents as required.
// Do now, as characters() may be called more than once
switch (nextDataType) {
case BOOL:
char first = value.charAt(0);
//thisStr = first == '0' ? "FALSE" : "TRUE";
//cellValue= new Boolean(first =='0' ? false: true);
cellValue=first == '0' ? "false" : "true";
break;
case ERROR:
//thisStr = "\"ERROR:" + value.toString() + '"';
cellValue=new String(value.toString());
break;
case FORMULA:
// A formula could result in a string value,
// so always add double-quote characters.
//thisStr = '"' + value.toString() + '"';
cellValue=new String(value.toString());
break;
case INLINESTR:
XSSFRichTextString rtsi = new XSSFRichTextString(value.toString());
//thisStr = '"' + rtsi.toString() + '"';
cellValue=new String(rtsi.toString());
break;
case SSTINDEX:
String sstIndex = value.toString();
try {
int idx = Integer.parseInt(sstIndex);
XSSFRichTextString rtss = new XSSFRichTextString(sharedStringsTable.getEntryAt(idx));
// thisStr = '"' + rtss.toString() + '"';
cellValue=new String(rtss.toString());
}
catch (NumberFormatException ex) {
System.out.println("Failed to parse SST index '" + sstIndex + "': " + ex.toString());
}
break;
case NUMBER:
String n = value.toString();
if (this.formatString != null && n.length() > 0){
cellValue = formatter.formatRawCellContents(Double.parseDouble(n), this.formatIndex, this.formatString);
//cellValue=new Double(Double.parseDouble(n));
}
else{
//thisStr = n;
cellValue=new String(n);
}
break;
default:
cellValue="";
//thisStr = "(TODO: Unexpected type: " + nextDataType + ")";
break;
}
// Output after we've seen the string contents
// Emit commas for any fields that were missing on this row
if (lastColumnNumber == -1) {
lastColumnNumber = 0;
}
// for (int i = lastColumnNumber; i < thisColumn; ++i){
// System.out.print(',');
// }
// Might be the empty string.
rowValues.put(cellCoordinate,cellValue);
//System.out.print(cellValue);
// Update column
if (thisColumn > -1)
lastColumnNumber = thisColumn;
} else if ("row".equals(name)) {
// We're onto a new row
//I have to pass new HashMap because otherwise all the map into archiveAcquisition have the same values
databaseAcquisitionServices.archiveAcquisition(new TreeMap<>(rowValues), rowCounter, acquisitionForm, sheetIndex);
//Clear the structure used to store row data
rowValues.clear();
rowCounter++;
//System.out.println();
lastColumnNumber = -1;
}
}
/**
* Captures characters only if a suitable element is open.
* Originally was just "v"; extended for inlineStr also.
*/
public void characters(char[] ch, int start, int length)
throws SAXException {
if (vIsOpen)
value.append(ch, start, length);
}
/**
* Converts an Excel column name like "C" to a zero-based index.
*
* #param name
* #return Index corresponding to the specified name
*/
private int nameToColumn(String name) {
int column = -1;
for (int i = 0; i < name.length(); ++i) {
int c = name.charAt(i);
column = (column + 1) * 26 + c - 'A';
}
return column;
}

You will likely need to count the number of rows that you see in a class derived from SheetContentsHandler, I don't think you need to have a separate XSSFSheetHAndler, I would rather use the default one and only have a dervied SheetContentsHandler which has callback-methods for all the things that you are interested in, e.g. startRow, endRow, cell.
You can take a look at the XLSX2CSV sample for an example of how streaming reading of XLSX files can be done, especially how the class SheetToCSV is used to get calls for each row/cell, you could e.g. use something like if (currentRow < startRow) { return } in the method cell() to skip rows.
The cell-method is called with a parameter String cellReference which can be used to retrieve the row/cell coordinates via new CellReference(cellReference).getCol() and new CellReference(cellReference).getRow().

Embed files into Excel using Apache POI

I am exporting data to Excel file using Apache POI. In one weird requirement, I need to embed one file in the Excel using this POI. I have the file, and can be taken into streams or as byte arrays. After googling for much time, I am in a doubt whether POI really supports my requirement. Can we embed files into Excel? :-(
Cheers,
Anoop

Ok, this took very long to finally work out, as there were a few things which didn't look very important at the beginning, but actually corrupted the file when they haven't been set right - especially in the Ole10Native wrapper, part of the unknown2 field actually contained the size (in bytes) of the following command string.
But first things first:
When you want to embed arbitrary files into one of the office formats, your best bet is to use the OLE 1.0 packager. It will be typically used, when you select insert->object from file.
So I've re-engineered an Excel 2003 file containing a PPT. As mentioned in my comment above, Excel will store its embedded objects in DirectoryNodes named "MBD....". In case of a Ole 1.0 Packager object, the interesting data will be found in the \1Ole10Native entry.
When you've inserted the data, you'll need to link it somehow in the Excel sheet. This is done by an EscherObject similar to a picture entry with additional attached records.
Apart from the many undocumented flags, there were a few things which puzzled me:
are the storage ids for the embedded objects just randomly assigned or is there some kind of number system?
I've searched for a more detailed description of the Ole10Native wrapper and especially for the ole 1.0 packager format, but apart of the M$ docu which sketchily handles it as one big byte chunk, most sources did some reengineering which looked very similar to the poi Ole10Native class ... of course the idea to check the libre office source came also to mind, but I have to admit the ones I've checked, only confused me :(
which one is the right clsid for the embedded object? ... i.e. for powerpoint there are quite a few. So if in doubt, obviously you'll need to lookup the clsid by a previously saved file from Office
Excel 2010 generates Biff8 files which embedded objects can't be opened by Libre Office!?!
the ole10Native object contains among other things a command line entry. would be interesting if someone can start other things than the embedded object with it ...
the BiffViewer crashed when I've used preview images bigger than some chunk size (~6kb). So either images would need to be chunked or the BiffViewer implementation is wrong ... this also caused some confusing for a while ...
Tested with POI 3.9, Libre Office 4.0, Office 2010 (I don't have Office 2003 anymore ...)
import java.awt.Color;
import java.io.*;
import java.lang.reflect.*;
import java.net.URL;
import javax.swing.ImageIcon;
import javax.swing.filechooser.FileSystemView;
import org.apache.poi.ddf.*;
import org.apache.poi.hpsf.ClassID;
import org.apache.poi.hslf.HSLFSlideShow;
import org.apache.poi.hslf.model.*;
import org.apache.poi.hslf.model.ShapeTypes;
import org.apache.poi.hslf.usermodel.SlideShow;
import org.apache.poi.hssf.dev.BiffViewer;
import org.apache.poi.hssf.model.*;
import org.apache.poi.hssf.record.*;
import org.apache.poi.hssf.usermodel.*;
import org.apache.poi.poifs.filesystem.*;
import org.apache.poi.ss.usermodel.*;
import org.apache.poi.util.*;
#SuppressWarnings("unused")
public class PoiOlePptInXls {
public static final OleType PACKAGE = new OleType("{0003000C-0000-0000-C000-000000000046}");
public static final OleType PPT_SHOW = new OleType("{64818D10-4F9B-11CF-86EA-00AA00B929E8}");
public static final OleType XLS_WORKBOOK = new OleType("{00020841-0000-0000-C000-000000000046}");
public static final OleType TXT_ONLY = new OleType("{5e941d80-bf96-11cd-b579-08002b30bfeb}"); // ???
static class OleType {
final String classId;
OleType(String classId) {
this.classId = classId;
}
ClassID getClassID() {
ClassID cls = new ClassID();
byte clsBytes[] = cls.getBytes();
String clsStr = classId.replaceAll("[{}-]", "");
for (int i=0; i<clsStr.length(); i+=2) {
clsBytes[i/2] = (byte)Integer.parseInt(clsStr.substring(i, i+2), 16);
}
return cls;
}
}
public static void main(String[] args) throws Exception {
POIFSFileSystem poifs = new POIFSFileSystem();
HSSFWorkbook wb = new HSSFWorkbook();
HSSFSheet sheet = wb.createSheet();
HSSFPatriarch patriarch = sheet.createDrawingPatriarch();
int previewIdxPpt = generatePreview(wb, "application/powerpoint");
int storageIdPpt = packageOleData(poifs, getSamplePPT(), "Example.ppt", "Example.ppt", "Example.ppt");
int previewIdxXls = generatePreview(wb, "application/excel");
int storageIdXls = packageOleData(poifs, getSampleXLS(), "Example.xls", "Example.xls", "Example.xls");
int previewIdxTxt = generatePreview(wb, "text/plain");
int storageIdTxt = packageOleData(poifs, getSampleTXT(), "Example.txt", "Example.txt", "Example.txt");
int rowoffset = 5;
int coloffset = 5;
CreationHelper ch = wb.getCreationHelper();
HSSFClientAnchor anchor = (HSSFClientAnchor)ch.createClientAnchor();
anchor.setAnchor((short)(2+coloffset), 1+rowoffset, 0, 0, (short)(3+coloffset), 5+rowoffset, 0, 0);
anchor.setAnchorType(ClientAnchor.DONT_MOVE_AND_RESIZE);
HSSFObjectData oleShape = createObjectData(poifs, storageIdPpt, 1, anchor, previewIdxPpt);
addShape(patriarch, oleShape);
anchor = (HSSFClientAnchor)ch.createClientAnchor();
anchor.setAnchor((short)(5+coloffset), 1+rowoffset, 0, 0, (short)(6+coloffset), 5+rowoffset, 0, 0);
anchor.setAnchorType(ClientAnchor.DONT_MOVE_AND_RESIZE);
oleShape = createObjectData(poifs, storageIdXls, 2, anchor, previewIdxXls);
addShape(patriarch, oleShape);
anchor = (HSSFClientAnchor)ch.createClientAnchor();
anchor.setAnchor((short)(3+coloffset), 10+rowoffset, 0, 0, (short)(5+coloffset), 11+rowoffset, 0, 0);
anchor.setAnchorType(ClientAnchor.DONT_MOVE_AND_RESIZE);
oleShape = createObjectData(poifs, storageIdTxt, 3, anchor, previewIdxTxt);
addShape(patriarch, oleShape);
anchor = (HSSFClientAnchor)ch.createClientAnchor();
anchor.setAnchor((short)(1+coloffset), -2+rowoffset, 0, 0, (short)(7+coloffset), 14+rowoffset, 0, 0);
anchor.setAnchorType(ClientAnchor.DONT_MOVE_AND_RESIZE);
HSSFSimpleShape circle = patriarch.createSimpleShape(anchor);
circle.setShapeType(HSSFSimpleShape.OBJECT_TYPE_OVAL);
circle.setNoFill(true);
poifs.getRoot().createDocument("Workbook", new ByteArrayInputStream(wb.getBytes()));
FileOutputStream fos = new FileOutputStream("ole_ppt_in_xls.xls");
poifs.writeFilesystem(fos);
fos.close();
}
static void addShape(HSSFPatriarch patriarch, HSSFShape shape) throws Exception {
patriarch.addShape(shape);
Method m = HSSFPatriarch.class.getDeclaredMethod("onCreate", HSSFShape.class);
m.setAccessible(true);
m.invoke(patriarch, shape);
}
static HSSFObjectData createObjectData(POIFSFileSystem poifs, int storageId, int objectIdx, HSSFClientAnchor anchor, int previewIdx) {
ObjRecord obj = new ObjRecord();
CommonObjectDataSubRecord ftCmo = new CommonObjectDataSubRecord();
ftCmo.setObjectType(CommonObjectDataSubRecord.OBJECT_TYPE_PICTURE);
ftCmo.setObjectId(objectIdx);
ftCmo.setLocked(true);
ftCmo.setPrintable(true);
ftCmo.setAutofill(true);
ftCmo.setAutoline(true);
ftCmo.setReserved1(0);
ftCmo.setReserved2(0);
ftCmo.setReserved3(0);
obj.addSubRecord(ftCmo);
obj.addSubRecord(SubRecord.createSubRecord(new LittleEndianByteArrayInputStream(new byte[]{7,0,2,0,2,0}), 0));
obj.addSubRecord(SubRecord.createSubRecord(new LittleEndianByteArrayInputStream(new byte[]{8,0,2,0,1,0}), 0));
EmbeddedObjectRefSubRecord ftPictFmla;
try {
Constructor<EmbeddedObjectRefSubRecord> con = EmbeddedObjectRefSubRecord.class.getDeclaredConstructor();
con.setAccessible(true);
ftPictFmla = con.newInstance();
} catch (Exception e) {
throw new RuntimeException("oops", e);
}
setField(ftPictFmla, "field_2_unknownFormulaData", new byte[]{2, 0, 0, 0, 0});
setField(ftPictFmla, "field_4_ole_classname", "Paket");
setField(ftPictFmla, "field_5_stream_id", (Integer)storageId);
obj.addSubRecord(ftPictFmla);
obj.addSubRecord(new EndSubRecord());
// create temporary picture, but don't attach it.
// It's neccessary to create the sp-container, which need to be minimal modified
// for oleshapes
HSSFPicture shape = new HSSFPicture(null, anchor);
EscherContainerRecord spContainer;
try {
Method m = HSSFPicture.class.getDeclaredMethod("createSpContainer");
m.setAccessible(true);
spContainer = (EscherContainerRecord)m.invoke(shape);
} catch (Exception e) {
throw new RuntimeException("oops", e);
}
EscherSpRecord spRecord = spContainer.getChildById(EscherSpRecord.RECORD_ID);
spRecord.setFlags(spRecord.getFlags() | EscherSpRecord.FLAG_OLESHAPE);
spRecord.setShapeType((byte)0x4B);
EscherOptRecord optRecord = spContainer.getChildById(EscherOptRecord.RECORD_ID);
EscherProperty ep = new EscherSimpleProperty(EscherProperties.BLIP__PICTUREID, false, false, 1);
optRecord.addEscherProperty(ep);
DirectoryEntry oleRoot;
try {
oleRoot = (DirectoryEntry)poifs.getRoot().getEntry(formatStorageId(storageId));
} catch (FileNotFoundException e) {
throw new RuntimeException("oops", e);
}
HSSFObjectData oleShape = new HSSFObjectData(spContainer, obj, oleRoot);
oleShape.setPictureIndex(previewIdx);
return oleShape;
}
static void setField(Object clazz, String fieldname, Object value) {
try {
Field f = clazz.getClass().getDeclaredField(fieldname);
f.setAccessible(true);
f.set(clazz, value);
} catch (Exception e) {
throw new RuntimeException("oops", e);
}
}
static void addOleStreamEntry(DirectoryEntry dir) throws IOException {
final String OLESTREAM_NAME = "\u0001Ole";
if (!dir.hasEntry(OLESTREAM_NAME)) {
// the following data was taken from an example libre office document
// beside this "\u0001Ole" record there were several other records, e.g. CompObj,
// OlePresXXX, but it seems, that they aren't neccessary
byte oleBytes[] = { 1, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
dir.createDocument(OLESTREAM_NAME, new ByteArrayInputStream(oleBytes));
}
}
static String formatStorageId(int storageId) {
return String.format("MBD%1$08X", storageId);
}
static int packageOleData(POIFSFileSystem poifs, byte oleData[], String label, String fileName, String command) throws IOException {
DirectoryNode root = poifs.getRoot();
// get free MBD-Node
int storageId = 0;
DirectoryEntry oleDir = null;
do {
String storageStr = formatStorageId(++storageId);
if (!root.hasEntry(storageStr)) {
oleDir = root.createDirectory(storageStr);
oleDir.setStorageClsid(PACKAGE.getClassID());
}
} while (oleDir == null);
addOleStreamEntry(oleDir);
Ole10Native2 oleNative = new Ole10Native2();
oleNative.setLabel(label);
oleNative.setFileName(fileName);
oleNative.setCommand(command);
oleNative.setDataBuffer(oleData);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
oleNative.writeOut(bos);
byte buf1[] = bos.toByteArray();
oleDir.createDocument(Ole10Native2.OLE10_NATIVE, new ByteArrayInputStream(buf1));
return storageId;
}
static byte[] getSamplePPT() {
HSLFSlideShow ss = HSLFSlideShow.create();
SlideShow ppt = new SlideShow(ss);
Slide slide = ppt.createSlide();
AutoShape sh1 = new AutoShape(ShapeTypes.Star32);
sh1.setAnchor(new java.awt.Rectangle(50, 50, 100, 200));
sh1.setFillColor(Color.red);
slide.addShape(sh1);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
try {
ppt.write(bos);
POIFSFileSystem poifs = new POIFSFileSystem(new ByteArrayInputStream(bos.toByteArray()));
poifs.getRoot().setStorageClsid(PPT_SHOW.getClassID());
bos.reset();
poifs.writeFilesystem(bos);
return bos.toByteArray();
} catch (IOException e) {
throw new RuntimeException("bla", e);
}
}
static byte[] getSampleXLS() {
HSSFWorkbook wb = new HSSFWorkbook();
HSSFSheet sheet = wb.createSheet();
sheet.createRow(5).createCell(2).setCellValue("yo dawg i herd you like embeddet objekts, so we put a ole in your ole so you can save a file while you save a file");
ByteArrayOutputStream bos = new ByteArrayOutputStream();
try {
wb.write(bos);
POIFSFileSystem poifs = new POIFSFileSystem(new ByteArrayInputStream(bos.toByteArray()));
poifs.getRoot().setStorageClsid(XLS_WORKBOOK.getClassID());
bos.reset();
poifs.writeFilesystem(bos);
return bos.toByteArray();
} catch (IOException e) {
throw new RuntimeException("bla", e);
}
}
static byte[] getSampleTXT() {
return "All your base are belong to us".getBytes();
}
/**
* to be defined, how to create a preview image for a start, I've taken just
* a dummy image, which will be replaced, when the user activates the ole
* object
*
* not really an alternativ:
* http://stackoverflow.com/questions/16704624/how-
* to-print-a-workbook-file-made-using-apache-poi-and-java
*
* #return image index of the preview image
*/
static int generatePreview(HSSFWorkbook workbook, String mimetype) {
try {
String url = "";
if ("application/powerpoint".equals(mimetype)) {
url = "http://upload.wikimedia.org/wikipedia/commons/thumb/a/a2/LibreOffice_Impress_icon_3.3.1_48_px.svg/40px-LibreOffice_Impress_icon_3.3.1_48_px.svg.png";
} else if ("application/excel".equals(mimetype)) {
url = "http://upload.wikimedia.org/wikipedia/commons/thumb/2/2d/LibreOffice_Calc_icon_3.3.1_48_px.svg/40px-LibreOffice_Calc_icon_3.3.1_48_px.svg.png";
} else if ("text/plain".equals(mimetype)) {
url = "http://upload.wikimedia.org/wikipedia/commons/thumb/a/a9/LibreOffice_Writer_icon_3.3.1_48_px.svg/40px-LibreOffice_Writer_icon_3.3.1_48_px.svg.png";
}
InputStream is = new URL(url).openStream();
byte previewImg[] = IOUtils.toByteArray(is);
is.close();
int pictIdx = workbook.addPicture(previewImg, HSSFWorkbook.PICTURE_TYPE_PNG);
return pictIdx;
} catch (IOException e) {
throw new RuntimeException("not really?", e);
}
}
/*
* Helper - determine length of zero terminated string (ASCIIZ).
*/
private static int getStringLength(byte[] data, int ofs) {
int len = 0;
while (len + ofs < data.length && data[ofs + len] != 0) {
len++;
}
len++;
return len;
}
}
The adapted Ole10Native class of POI with write-support:
import java.io.*;
import org.apache.poi.poifs.filesystem.*;
import org.apache.poi.util.*;
/**
* Represents an Ole10Native record which is wrapped around certain binary files
* being embedded in OLE2 documents.
*
* #author Rainer Schwarze
*/
public class Ole10Native2 {
public static final String OLE10_NATIVE = "\u0001Ole10Native";
protected static final String ISO1 = "ISO-8859-1";
// (the fields as they appear in the raw record:)
protected int totalSize; // 4 bytes, total size of record not including this
// field
protected short flags1 = 2; // 2 bytes, unknown, mostly [02 00]
protected String label; // ASCIIZ, stored in this field without the
// terminating zero
protected String fileName; // ASCIIZ, stored in this field without the
// terminating zero
protected short flags2 = 0; // 2 bytes, unknown, mostly [00 00]
protected short unknown1 = 3;
protected String command; // ASCIIZ, stored in this field without the
// terminating zero
protected byte[] dataBuffer; // varying size, the actual native data
protected short flags3 = 0; // some final flags? or zero terminators?,
// sometimes not there
/**
* Creates an instance of this class from an embedded OLE Object. The OLE
* Object is expected to include a stream "{01}Ole10Native" which
* contains the actual data relevant for this class.
*
* #param poifs
* POI Filesystem object
* #return Returns an instance of this class
* #throws IOException
* on IO error
* #throws Ole10NativeException
* on invalid or unexcepted data format
*/
public static Ole10Native2 createFromEmbeddedOleObject(POIFSFileSystem poifs) throws IOException, Ole10NativeException {
return createFromEmbeddedOleObject(poifs.getRoot());
}
/**
* Creates an instance of this class from an embedded OLE Object. The OLE
* Object is expected to include a stream "{01}Ole10Native" which
* contains the actual data relevant for this class.
*
* #param directory
* POI Filesystem object
* #return Returns an instance of this class
* #throws IOException
* on IO error
* #throws Ole10NativeException
* on invalid or unexcepted data format
*/
public static Ole10Native2 createFromEmbeddedOleObject(DirectoryNode directory) throws IOException, Ole10NativeException {
boolean plain = false;
try {
directory.getEntry("\u0001Ole10ItemName");
plain = true;
} catch (FileNotFoundException ex) {
plain = false;
}
DocumentEntry nativeEntry = (DocumentEntry) directory.getEntry(OLE10_NATIVE);
byte[] data = new byte[nativeEntry.getSize()];
directory.createDocumentInputStream(nativeEntry).read(data);
return new Ole10Native2(data, 0, plain);
}
/**
* Creates an instance and fills the fields based on the data in the given
* buffer.
*
* #param data
* The buffer containing the Ole10Native record
* #param offset
* The start offset of the record in the buffer
* #throws Ole10NativeException
* on invalid or unexcepted data format
*/
public Ole10Native2(byte[] data, int offset) throws Ole10NativeException {
this(data, offset, false);
}
/**
* Creates an instance and fills the fields based on the data in the given
* buffer.
*
* #param data
* The buffer containing the Ole10Native record
* #param offset
* The start offset of the record in the buffer
* #param plain
* Specified 'plain' format without filename
* #throws Ole10NativeException
* on invalid or unexcepted data format
*/
public Ole10Native2(byte[] data, int offset, boolean plain) throws Ole10NativeException {
int ofs = offset; // current offset, initialized to start
if (data.length < offset + 2) {
throw new Ole10NativeException("data is too small");
}
totalSize = LittleEndian.getInt(data, ofs);
ofs += LittleEndianConsts.INT_SIZE;
if (plain) {
dataBuffer = new byte[totalSize - 4];
System.arraycopy(data, 4, dataBuffer, 0, dataBuffer.length);
int dataSize = totalSize - 4;
byte[] oleLabel = new byte[8];
System.arraycopy(dataBuffer, 0, oleLabel, 0, Math.min(dataBuffer.length, 8));
label = "ole-" + HexDump.toHex(oleLabel);
fileName = label;
command = label;
} else {
flags1 = LittleEndian.getShort(data, ofs);
ofs += LittleEndianConsts.SHORT_SIZE;
int len = getStringLength(data, ofs);
label = StringUtil.getFromCompressedUnicode(data, ofs, len - 1);
ofs += len;
len = getStringLength(data, ofs);
fileName = StringUtil.getFromCompressedUnicode(data, ofs, len - 1);
ofs += len;
flags2 = LittleEndian.getShort(data, ofs);
ofs += LittleEndianConsts.SHORT_SIZE;
unknown1 = LittleEndian.getShort(data, ofs);
ofs += LittleEndianConsts.SHORT_SIZE;
len = LittleEndian.getInt(data, ofs);
ofs += LittleEndianConsts.INT_SIZE;
command = StringUtil.getFromCompressedUnicode(data, ofs, len - 1);
ofs += len;
if (totalSize < ofs) {
throw new Ole10NativeException("Invalid Ole10Native");
}
int dataSize = LittleEndian.getInt(data, ofs);
ofs += LittleEndianConsts.INT_SIZE;
if (dataSize < 0 || totalSize - (ofs - LittleEndianConsts.INT_SIZE) < dataSize) {
throw new Ole10NativeException("Invalid Ole10Native");
}
dataBuffer = new byte[dataSize];
System.arraycopy(data, ofs, dataBuffer, 0, dataSize);
ofs += dataSize;
// if (unknown1.length > 0) {
// flags3 = LittleEndian.getShort(data, ofs);
// ofs += LittleEndianConsts.SHORT_SIZE;
// } else {
// flags3 = 0;
// }
}
}
public Ole10Native2() {}
/*
* Helper - determine length of zero terminated string (ASCIIZ).
*/
private static int getStringLength(byte[] data, int ofs) {
int len = 0;
while (len + ofs < data.length && data[ofs + len] != 0) {
len++;
}
len++;
return len;
}
/**
* Returns the value of the totalSize field - the total length of the
* structure is totalSize + 4 (value of this field + size of this field).
*
* #return the totalSize
*/
public int getTotalSize() {
return totalSize;
}
/**
* Returns flags1 - currently unknown - usually 0x0002.
*
* #return the flags1
*/
public short getFlags1() {
return flags1;
}
/**
* Returns the label field - usually the name of the file (without
* directory) but probably may be any name specified during
* packaging/embedding the data.
*
* #return the label
*/
public String getLabel() {
return label;
}
/**
* Returns the fileName field - usually the name of the file being embedded
* including the full path.
*
* #return the fileName
*/
public String getFileName() {
return fileName;
}
/**
* Returns flags2 - currently unknown - mostly 0x0000.
*
* #return the flags2
*/
public short getFlags2() {
return flags2;
}
/**
* Returns unknown1 field - currently unknown.
*
* #return the unknown1
*/
public short getUnknown1() {
return unknown1;
}
/**
* Returns the unknown2 field - currently being a byte[3] - mostly {0, 0,
* 0}.
*
* #return the unknown2
*/
// public short getUnknown2() {
// return unknown2;
// }
/**
* Returns the command field - usually the name of the file being embedded
* including the full path, may be a command specified during embedding the
* file.
*
* #return the command
*/
public String getCommand() {
return command;
}
/**
* Returns the size of the embedded file. If the size is 0 (zero), no data
* has been embedded. To be sure, that no data has been embedded, check
* whether {#link #getDataBuffer()} returns <code>null</code>.
*
* #return the dataSize
*/
public int getDataSize() {
return dataBuffer.length;
}
/**
* Returns the buffer containing the embedded file's data, or
* <code>null</code> if no data was embedded. Note that an embedding may
* provide information about the data, but the actual data is not included.
* (So label, filename etc. are available, but this method returns
* <code>null</code>.)
*
* #return the dataBuffer
*/
public byte[] getDataBuffer() {
return dataBuffer;
}
/**
* Returns the flags3 - currently unknown.
*
* #return the flags3
*/
public short getFlags3() {
return flags3;
}
/**
* Have the contents printer out into an OutputStream, used when writing a
* file back out to disk (Normally, atom classes will keep their bytes
* around, but non atom classes will just request the bytes from their
* children, then chuck on their header and return)
*/
public void writeOut(OutputStream out) throws IOException {
byte intbuf[] = new byte[LittleEndianConsts.INT_SIZE];
byte shortbuf[] = new byte[LittleEndianConsts.SHORT_SIZE];
byte bytebuf[] = new byte[LittleEndianConsts.BYTE_SIZE];
// LittleEndian.putInt(_header, 4, _data.length);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bos.write(intbuf); // total size, will be determined later ..
LittleEndian.putShort(shortbuf, 0, getFlags1());
bos.write(shortbuf);
bos.write(getLabel().getBytes(ISO1));
bos.write(0);
bos.write(getFileName().getBytes(ISO1));
bos.write(0);
LittleEndian.putShort(shortbuf, 0, getFlags2());
bos.write(shortbuf);
LittleEndian.putShort(shortbuf, 0, getUnknown1());
bos.write(shortbuf);
LittleEndian.putInt(intbuf, 0, getCommand().length()+1);
bos.write(intbuf);
bos.write(getCommand().getBytes(ISO1));
bos.write(0);
LittleEndian.putInt(intbuf, 0, getDataBuffer().length);
bos.write(intbuf);
bos.write(getDataBuffer());
LittleEndian.putShort(shortbuf, 0, getFlags3());
bos.write(shortbuf);
// update total size - length of length-field (4 bytes)
byte data[] = bos.toByteArray();
totalSize = data.length - LittleEndianConsts.INT_SIZE;
LittleEndian.putInt(data, 0, totalSize);
out.write(data);
}
public void setFlags1(short flags1) {
this.flags1 = flags1;
}
public void setFlags2(short flags2) {
this.flags2 = flags2;
}
public void setFlags3(short flags3) {
this.flags3 = flags3;
}
public void setLabel(String label) {
this.label = label;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public void setCommand(String command) {
this.command = command;
}
public void setUnknown1(short unknown1) {
this.unknown1 = unknown1;
}
// public void setUnknown2(short unknown2) {
// this.unknown2 = unknown2;
// }
public void setDataBuffer(byte dataBuffer[]) {
this.dataBuffer = dataBuffer;
}
}

This page of the online documentation might answer your question:
http://poi.apache.org/components/poifs/embeded.html

ObjectInputStream: is this correct way to unblock

ObjectInputStream blocks when created until it recieves a serial input stream ans verifies it. I was trying to make my first program using sockets through it and found this. I used a dummy object so that it doesn't block. The code is here:
import java.io.*;
import java.net.*;
import java.util.*;
class Dummy implements Serializable {
}
class X_Int implements Serializable {
int x;
}
class Server {
public static void main(String args[]) throws Exception {
ServerSocket ss = new ServerSocket(5879);
Socket client = ss.accept();
ObjectOutputStream out = new ObjectOutputStream(client.getOutputStream());
out.writeObject(new Dummy());
ObjectInputStream in = new ObjectInputStream(client.getInputStream());
in.readObject();
out.flush();
out.writeObject(new Date());
out.flush();
out.close();
}
}
class Client {
public static void main(String args[]) throws Exception {
Socket server = new Socket("localhost", 5879);
ObjectOutputStream out = new ObjectOutputStream(server.getOutputStream());
out.writeObject(new Dummy());
ObjectInputStream in = new ObjectInputStream(server.getInputStream());
in.readObject();
out.flush();
Date d = (Date)in.readObject();
System.out.println(d);
}
}
Is this the right way. Please comment.

You just need to flush() the output before creating the object input stream.
You don't need to send dummy objects.

A better way is to get rid of the cause of blocking in the first place. Use these classes instead on both ends, if you can:
public class ObjInputStream extends ObjectInputStream {
/**
* #param in
* #throws IOException
*/
public ObjInputStream(InputStream in) throws IOException {
super(in);
}
/* (non-Javadoc)
* #see java.io.ObjectInputStream#readStreamHeader()
*/
#Override
protected void readStreamHeader() throws IOException, StreamCorruptedException {
}
}
and
public class ObjOutputStream extends ObjectOutputStream {
/**
* #param out
* #throws IOException
*/
public ObjOutputStream(OutputStream out) throws IOException {
super(out);
}
/* (non-Javadoc)
* #see java.io.ObjectOutputStream#writeStreamHeader()
*/
#Override
protected void writeStreamHeader() throws IOException {
}
}
This removes the functions which are called to ascertain stream version info and such.
Additionally, as you are using TCP packets, IP fragmentation will cause your objects not be received 'whole' on the other end -- TCP is a stream socket. What you need is an additional framing input / output class. Luckily, I already coded this :)
/**
*
*/
package objtest;
import java.io.IOException;
import java.io.InputStream;
import java.nio.BufferUnderflowException;
import java.nio.ByteBuffer;
import java.util.ArrayDeque;
import java.util.Queue;
import kokuks.KokuKS;
/**
* UnrealConceptTest - FramedInputStream
* #version 1.0
*/
public class FramedInputStream extends InputStream {
public static final int INITIAL_BUFFER_SIZE = 8 << 1;
public static final int FRAME_HEADER_1 = 0xBEEFFACE;
public static final int FRAME_HEADER_2 = 0xFACEBEEF;
public static final byte[] HEADER_BYTES = new byte[4 * 2];
protected static final byte[] CURR_HEADER_BUFF = new byte[HEADER_BYTES.length];
static {
ByteBuffer b = ByteBuffer.allocateDirect(8);
b.putInt(FRAME_HEADER_1);
b.putInt(FRAME_HEADER_2);
ByteBuffer b2 = (ByteBuffer) b.flip();
b2.get(HEADER_BYTES, 0, 4);
b2.get(HEADER_BYTES, 3, 4);
}
protected int size = 0;
protected int chain = 0;
protected boolean inFrame = false;
protected boolean readingSize = false;
protected int sizePos = 0;
protected int dbgput = 0;
protected ByteBuffer bb = ByteBuffer.allocateDirect(INITIAL_BUFFER_SIZE);
protected Queue<ByteBuffer> bbq = new ArrayDeque<ByteBuffer>();
protected ByteBuffer currBuff = null;
protected final boolean recoverFromError;
/**
*
*/
public FramedInputStream(boolean recoverFromError) {
this.recoverFromError = recoverFromError;
}
public FramedInputStream() {
this(true);
}
protected boolean ensureFramebufferCapacity(int min) {
int mymin = 1 << min;
if (mymin <= bb.capacity()) return false;
int num = bb.capacity();
while (num < mymin) num <<= 1;
ByteBuffer bb2 = ByteBuffer.allocateDirect(num);
// copy old data into new bytebuffer
int bb_pos = bb.position();
bb.rewind();
bb2.put(bb);
bb = bb2;
if (KokuKS.DEBUG_MODE) System.out.println("modified buffer size to: " + num);
return true;
}
/**
* #return the recoverFromError
*/
public boolean isRecoverFromError() {
return recoverFromError;
}
/* (non-Javadoc)
* #see java.io.InputStream#read()
*/
#Override
public int read() throws IOException {
if (currBuff == null || !currBuff.hasRemaining()) return -1;
byte b = currBuff.get();
//System.out.println("data: " + b);
return b;
}
public void putBuffer(ByteBuffer source) {
ensureFramebufferCapacity(bb.capacity() + source.remaining());
while (source.hasRemaining()) {
putByte(source.get());
}
}
public boolean checkCompleteFrame() {
return !bbq.isEmpty();
}
/* (non-Javadoc)
* #see java.io.InputStream#available()
*/
#Override
public int available() throws IOException {
return currBuff != null ? currBuff.remaining() : 0;
}
public int read(byte[] data) {
if (currBuff == null || !currBuff.hasRemaining()) {
return -1;
}
if (data.length > currBuff.remaining()) {
throw new BufferUnderflowException();
}
currBuff.get(data);
//System.out.println("data: " + new String(data));
return data.length;
}
public boolean nextFrame() {
ByteBuffer bbf = bbq.poll();
if (bbf != null) {
/*
System.out.println("bbf limit: " + bbf.limit());
System.out.println("bbf pos: " + bbf.position());
System.out.println("bbf data: " + new String(bbf.array()));
*/
//byte[] data = bbf.array();
//for (int i = 0; i < data.length; i++) {
// byte by = data[i];
// System.out.println("b: " + (by > 32 ? new String(new byte[] {by}) : "??") + ", " + by);
//}
currBuff = ByteBuffer.allocateDirect(bbf.limit());
currBuff.put(bbf).flip();
bbf.rewind();
/*
System.out.println("currbuf limit: " + currBuff.limit());
System.out.println("currbuf pos: " + currBuff.position());
System.out.println("currbuf data: " + new String(currBuff.array()));
*/
currBuff.rewind();
currBuff.position(1);
return true;
}
return false;
}
public void putByte(byte b) {
//System.out.println("pb b: " + ObjTest.getByteStr(b));
if (recoverFromError || !inFrame) {
if (b == HEADER_BYTES[chain++]) {
if (chain >= (HEADER_BYTES.length)) {
if (KokuKS.DEBUG_MODE) System.out.println("got header!" + (inFrame ? " (recovered)" : ""));
// we have a header! hurrah.
inFrame = true;
sizePos = 0;
size = 0;
readingSize = true;
chain = 0;
bb.clear();
}
} else {
chain = 0;
}
}
if (inFrame) {
if (readingSize) {
size += (b & 0xFF) << ((8 * 3) - (8 * sizePos));
//System.out.println("new size: " + size);
sizePos++;
if (sizePos >= 4) {
// we've read the size :)
readingSize = false;
sizePos = 0;
ensureFramebufferCapacity(size);
bb.clear();
bb.limit(size); // set buffer limit to size
//System.out.println("bb limit set to: " + bb.limit());
}
} else {
//System.out.println("put: " + dbgput++ + ", " + ObjTest.getByteStr(b));
bb.put(b);
if (!bb.hasRemaining()) {
bb.flip();
//System.out.println("bb limit after flip(): " + bb.limit());
//System.out.println("bblimit: " + bb.limit());
ByteBuffer newbuf = ByteBuffer.allocateDirect(bb.limit());
newbuf.put(bb).flip(); //we have to flip this
bbq.offer(newbuf);
//byte[] data = newbuf.array();
//for (int i = 0; i < newbuf.limit(); i++) {
// byte by = data[i];
// System.out.println("b: " + (by > 32 ? new String(new byte[] {by}) : "??") + ", " + by);
//}
inFrame = false;
readingSize = false;
size = 0;
sizePos = 0;
chain = 0;
bb.clear();
if (KokuKS.DEBUG_MODE) System.out.println("FIS: complete object");
//System.out.println("FIS: newbuf: " + new String(newbuf.array(), 0, newbuf.limit()));
}
}
}
}
}
and
/**
*
*/
package objtest;
import java.io.IOException;
import java.nio.ByteBuffer;
import koku.util.io.ByteBufferOutputStream;
/**
* UnrealConceptTest - FramedOutputStream
* #version 1.0
* #author Chris Dennett
*/
public class FramedOutputStream extends ByteBufferOutputStream {
public static final int FRAME_HEADER_1 = 0xBEEFFACE;
public static final int FRAME_HEADER_2 = 0xFACEBEEF;
public static final byte[] HEADER_BYTES = new byte[4 * 2];
public static final byte[] CURR_HEADER_BUFF = new byte[HEADER_BYTES.length];
/* We pad the beginning of our buffer so that we can write the frame
* length when the time comes. */
protected static final byte[] SIZE_PAD = new byte[4];
static {
ByteBuffer b = ByteBuffer.allocate(8);
b.putInt(FRAME_HEADER_1);
b.putInt(FRAME_HEADER_2);
ByteBuffer b2 = (ByteBuffer) b.flip();
b2.get(HEADER_BYTES, 0, 4);
b2.get(HEADER_BYTES, 3, 4);
}
/**
*
*/
public FramedOutputStream() {
try {
write(HEADER_BYTES);
write(SIZE_PAD);
} catch (IOException e) {
System.out.println("Couldn't write header padding!");
}
}
/* (non-Javadoc)
* #see koku.util.io.ByteBufferOutputStream#flip()
*/
#Override
public ByteBuffer flip() {
// flip the buffer which will limit it to it's current position
super.flip();
// then write the frame length and rewind back to the start of the
// buffer so that all the data is available
_buffer.position(11);
int size = _buffer.remaining();
//System.out.println("remaining after complete header: " + size);
_buffer.position(7);
//System.out.println("remaining after frameheader: " + _buffer.remaining());
putSizeAsBytes(size, _buffer);
//System.out.println("written size: " + size);
// System.out.println("buffer limit: " + _buffer.limit());
//System.out.println("_buffer: " + new String( _buffer.array(), 0, _buffer.limit()));
_buffer.position(11);
// System.out.println("_buffer11: " + ObjTest.getByteStr(_buffer.get()));
//System.out.println("_buffer12: " + ObjTest.getByteStr(_buffer.get()));
//System.out.println("_buffer13: " + ObjTest.getByteStr(_buffer.get()));
//System.out.println("_buffer14: " + ObjTest.getByteStr(_buffer.get()));
_buffer.rewind();
//_buffer.rewind();
//while (_buffer.hasRemaining()) {
// byte b = _buffer.get();
// System.out.println("b: " + (b > 32 ? new String(new byte[] {b}) : "??") + ", " + b);
//}
_buffer.rewind();
return _buffer;
}
/* (non-Javadoc)
* #see koku.util.io.ByteBufferOutputStream#reset()
*/
#Override
public void reset() {
super.reset();
try {
write(HEADER_BYTES);
write(SIZE_PAD);
} catch (IOException e) {
System.out.println("Couldn't write header padding!");
}
}
public static void putSizeAsBytes(int size, ByteBuffer bb) {
//System.out.println("putSizeAsBytes: given size: " + size);
// encode
for (int i = 0; i < 4; i++) {
bb.put((byte)((size >>> ((8 * 3) - (8 * i))) & 0xFF));
}
}
}
BBOS:
//
// $Id: ByteBufferOutputStream.java 5829 2009-06-20 21:09:34Z mdb $
//
// Narya library - tools for developing networked games
// Copyright (C) 2002-2009 Three Rings Design, Inc., All Rights Reserved
// http://www.threerings.net/code/narya/
//
// This library is free software; you can redistribute it and/or modify it
// under the terms of the GNU Lesser General Public License as published
// by the Free Software Foundation; either version 2.1 of the License, or
// (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public
// License along with this library; if not, write to the Free Software
// Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
package misc;
import java.io.OutputStream;
import java.nio.BufferOverflowException;
import java.nio.ByteBuffer;
import org.apache.mina.core.buffer.IoBuffer;
/**
* Stores output in an {#link ByteBuffer} that grows automatically to accommodate the data.
*/
public class ByteBufferOutputStream extends OutputStream
{
/**
* Creates a new byte buffer output stream.
*/
public ByteBufferOutputStream ()
{
_buffer = IoBuffer.allocate(INITIAL_BUFFER_SIZE);
}
/**
* Returns a reference to the underlying buffer.
*/
public IoBuffer getBuffer ()
{
return _buffer;
}
/**
* Flips and returns the buffer. The returned buffer will have a position of zero and a limit
* equal to the number of bytes written. Call {#link #reset} to reset the buffer before
* writing again.
*/
public IoBuffer flip ()
{
return _buffer.flip();
}
/**
* Resets our internal buffer.
*/
public void reset ()
{
_buffer.clear();
}
#Override // documentation inherited
public void write (int b)
{
try {
_buffer.put((byte)b);
} catch (BufferOverflowException boe) {
expand(1);
_buffer.put((byte)b);
}
}
#Override // documentation inherited
public void write (byte[] b, int off, int len)
{
// sanity check the arguments
if ((off < 0) || (off > b.length) || (len < 0) ||
((off + len) > b.length) || ((off + len) < 0)) {
throw new IndexOutOfBoundsException();
} else if (len == 0) {
return;
}
try {
_buffer.put(b, off, len);
} catch (BufferOverflowException boe) {
expand(len);
_buffer.put(b, off, len);
}
}
/**
* Expands our buffer to accomodate the specified capacity.
*/
protected final void expand (int needed)
{
_buffer.expand(needed);
}
/** The buffer in which we store our frame data. */
protected IoBuffer _buffer;
/** The default initial size of the internal buffer. */
protected static final int INITIAL_BUFFER_SIZE = 32;
}