We Keep Coding

vector serialization in mapreduce

How do I serialize Vector in mapreduce.Below is my code where Adjacent_nodes is the vector to be written and read from Data streammers
#Override
public void readFields(DataInput in) throws IOException {
VID=in.readLong();
Adjacent_nodes=in.read<????>();
}
#Override
public void write(DataOutput out) throws IOException {
out.writeLong(VID);
out.write<???>(Adjacent_nodes);
}

#Override
public void readFields(DataInput in) throws IOException {
VID=in.readLong();
long vector_size=in.readLong();
for(int i=0;i<count;i++)
Adjacent.addElement(in.readLong());
}
}
#Override
public void write(DataOutput out) throws IOException {
out.writeLong(VID);
out.writeLong(Adjacent_nodes.size());
for(int i=0;i<count;i++)
out.writeLong(Adjacent.get(i));
}

Java Zip File System Provider: ReadOnly on remote drive [Windows]

I have a problem with Zip File System Provider: If the zip file is on a remote drive (mapped or not seems to be irrelevant), the virtual file system is readonly, although the file itself is not. I wrote a minimal sample code:
public static void main(String[] args) throws IOException {
File workingDir = new File(args[0]);
File source = new File(workingDir, "in.zip");
File target = new File(workingDir, "out.zip");
Files.copy(source.toPath(), target.toPath(), StandardCopyOption.REPLACE_EXISTING);
try (FileSystem zipfs = FileSystems.newFileSystem(target.toPath(), null)) {
Path pathInZipfile = zipfs.getPath("test.xml");
System.out.println("zipfile writable: " + target.canWrite());
System.out.println("zipFS writable: " + !zipfs.isReadOnly());
Files.delete(pathInZipfile);
System.out.println("File successfully deleted");
} catch (IOException e) {
e.printStackTrace();
}
}
If workingDir is a local directory, everything works fine. However, if it is a (mapped) remote drive, i get:
zipfile writable: true
zipFS writable: false
Exception in thread "main" java.nio.file.ReadOnlyFileSystemException
at com.sun.nio.zipfs.ZipFileSystem.checkWritable(ZipFileSystem.java:155)
at com.sun.nio.zipfs.ZipFileSystem.deleteFile(ZipFileSystem.java:1335)
at com.sun.nio.zipfs.ZipPath.delete(ZipPath.java:655)
at com.sun.nio.zipfs.ZipFileSystemProvider.delete(ZipFileSystemProvider.java:206)
at java.nio.file.Files.delete(Unknown Source)
at zipfs.ZipFS.main(ZipFS.java:23)
Am I doing something wrong? Is it impossible? Is there a workaround?

I ran into the same thing and I looked at the JDK code.
Findings
In ZipFileSystem.java there are three relevant lines:
zfpath.getFileSystem().provider().checkAccess(zfpath, AccessMode.READ);
if (!Files.isWritable(zfpath))
this.readOnly = true;
zfPath is a Path object. Something in the Windows FileSystem provider blocks write access to a zip archive path. Doesn't seem like there is much to be done about that.
Workaround
What I used as a workaround was to:
Create the zip archive in a temp folder
Populate the zip archive
Copy the temp file to the original mapped drive location
As long as the mapped drive is writable in contexts outside of the zip FileSystem, this method works.

We run in exactly the same problem, and have deviced somewhat barbarian solution.
FileSystems.newFileSystem(Path) accepts Path interface, so it's possible to satisfy it wrapping real Path instance.
Here is the solution that implements required FileSysteProvider.checkAccess():
private static FileSystem createZip(Path zipPath)
throws Exception
{
var fileSystem = zipPath.getFileSystem();
var provider = fileSystem.provider();
return FileSystems.newFileSystem(
new Path()
{
private Path path(Path path)
{
return this == path ? zipPath : path;
}
#Override
public FileSystem getFileSystem()
{
return new FileSystem()
{
public Set<String> supportedFileAttributeViews()
{
return fileSystem.supportedFileAttributeViews();
}
#Override
public FileSystemProvider provider()
{
return new FileSystemProvider()
{
#Override
public void setAttribute(
Path path,
String attribute,
Object value,
LinkOption... options)
throws IOException
{
provider.setAttribute(path(path), attribute, value, options);
}
#Override
public Map<String, Object> readAttributes(
Path path,
String attributes,
LinkOption... options)
throws IOException
{
return provider.
readAttributes(path(path), attributes, options);
}
#Override
public <A extends BasicFileAttributes> A readAttributes(
Path path,
Class<A> type,
LinkOption... options)
throws IOException
{
return provider.readAttributes(path(path), type, options);
}
#Override
public FileSystem newFileSystem(URI uri, Map<String, ?> env)
throws IOException
{
return provider.newFileSystem(uri, env);
}
#Override
public DirectoryStream<Path> newDirectoryStream(
Path dir,
Filter<? super Path> filter)
throws IOException
{
return provider.newDirectoryStream(dir, filter);
}
#Override
public SeekableByteChannel newByteChannel(
Path path,
Set<? extends OpenOption> options,
FileAttribute<?>... attrs)
throws IOException
{
return provider.newByteChannel(path(path), options, attrs);
}
#Override
public void move(
Path source,
Path target,
CopyOption... options)
throws IOException
{
provider.move(path(source), path(target), options);
}
#Override
public boolean isSameFile(Path path, Path path2)
throws IOException
{
return provider.isSameFile(path(path), path(path2));
}
#Override
public boolean isHidden(Path path)
throws IOException
{
return provider.isHidden(path(path));
}
#Override
public String getScheme()
{
return provider.getScheme();
}
#Override
public Path getPath(URI uri)
{
return provider.getPath(uri);
}
#Override
public FileSystem getFileSystem(URI uri)
{
return provider.getFileSystem(uri);
}
#Override
public FileStore getFileStore(Path path)
throws IOException
{
return provider.getFileStore(path(path));
}
#Override
public <V extends FileAttributeView> V getFileAttributeView(
Path path,
Class<V> type,
LinkOption... options)
{
return provider.
getFileAttributeView(path(path), type, options);
}
#Override
public void delete(Path path)
throws IOException
{
provider.delete(path(path));
}
#Override
public void createDirectory(Path dir, FileAttribute<?>... attrs)
throws IOException
{
provider.createDirectory(path(dir), attrs);
}
#Override
public void copy(
Path source,
Path target,
CopyOption... options)
throws IOException
{
provider.copy(path(source), path(target), options);
}
#Override
public void checkAccess(Path path, AccessMode... modes)
throws IOException
{
if ((modes != null) &&
(modes.length == 1) &&
(modes[0] == AccessMode.WRITE))
{
return;
}
provider.checkAccess(path(path), modes);
}
};
}
#Override
public WatchService newWatchService()
throws IOException
{
return fileSystem.newWatchService();
}
#Override
public boolean isReadOnly()
{
return false;
}
#Override
public boolean isOpen()
{
return fileSystem.isOpen();
}
#Override
public UserPrincipalLookupService getUserPrincipalLookupService()
{
return fileSystem.getUserPrincipalLookupService();
}
#Override
public String getSeparator()
{
return fileSystem.getSeparator();
}
#Override
public Iterable<Path> getRootDirectories()
{
return fileSystem.getRootDirectories();
}
#Override
public PathMatcher getPathMatcher(String syntaxAndPattern)
{
return fileSystem.getPathMatcher(syntaxAndPattern);
}
#Override
public Path getPath(String first, String... more)
{
return fileSystem.getPath(first, more);
}
#Override
public Iterable<FileStore> getFileStores()
{
return fileSystem.getFileStores();
}
#Override
public void close() throws IOException
{
fileSystem.close();
}
};
}
#Override
public boolean isAbsolute()
{
return zipPath.isAbsolute();
}
#Override
public Path getRoot()
{
return zipPath.getRoot();
}
#Override
public Path getFileName()
{
return zipPath.getFileName();
}
#Override
public Path getParent()
{
return zipPath.getParent();
}
#Override
public int getNameCount()
{
return zipPath.getNameCount();
}
#Override
public Path getName(int index)
{
return zipPath.getName(index);
}
#Override
public Path subpath(int beginIndex, int endIndex)
{
return zipPath.subpath(beginIndex, endIndex);
}
#Override
public boolean startsWith(Path other)
{
return zipPath.startsWith(other);
}
#Override
public boolean endsWith(Path other)
{
return zipPath.endsWith(other);
}
#Override
public Path normalize()
{
return zipPath.normalize();
}
#Override
public Path resolve(Path other)
{
return zipPath.relativize(other);
}
#Override
public Path relativize(Path other)
{
return zipPath.relativize(other);
}
#Override
public URI toUri()
{
return zipPath.toUri();
}
#Override
public Path toAbsolutePath()
{
return zipPath.toAbsolutePath();
}
#Override
public Path toRealPath(LinkOption... options)
throws IOException
{
return zipPath.toRealPath(options);
}
#Override
public WatchKey register(
WatchService watcher,
Kind<?>[] events,
Modifier... modifiers)
throws IOException
{
return zipPath.register(watcher, events, modifiers);
}
#Override
public int compareTo(Path other)
{
return zipPath.compareTo(other);
}
},
Map.of("create", "true"));
}
This is more like a hack but we think it solves the bug in original ZipFileSystem

Try setting the private field readOnly in ZipFileSystem to false with reflection.

adding namespace attribute to non-root elements using jaxb [duplicate]

I am trying to marshall a message using the following snippet:
JAXBContext jContext = JAXBContext.newInstance(Iq.class);
Marshaller m = newJAXBContext.createMarshaller();
m.setProperty(Marshaller.JAXB_FRAGMENT, Boolean.TRUE);
m.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);
Bind bind = new Bind();
bind.setResource("resource");
Iq iq = new Iq();
iq.setId(iqId);
iq.setType("set");
iq.getAnies().add(bind);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
m.marshal(iq, baos);
Here, Iq and Bind are the objects formed from the relevant xmpp schemas. My problem is, with jaxb 2.0 and later versions, all the namespaces are declared in the root element:
<iq from='juliet#example.com/balcony'
id='rg1'
type='get' xmlns='jabber:client' xmlns:ns1='urn:ietf:params:xml:ns:xmpp-bind'>
<ns1:bind>
<ns1:resource>resource</ns1:resource>
</ns1:bind>
</iq>
But, what is needed here is that the namespaces should occupy the appropriate places:
<iq from='juliet#example.com/balcony'
id="rg1"
type="get" xmlns="jabber:client">
<bind xmlns="urn:ietf:params:xml:ns:xmpp-bind">
<resource>resource</resource>
</bind>
</iq>
Is there a way to marshall the xmpp stanzas as you see them in the 2nd xml stanza through JAXB 2.0 or later versions?
Long story short, I have 2 problems here:
1. Declaring the namespaces at appropriate locations.
2. removing the namespace prefix which I understand can be removed using the NamespacePrefixMapper? Not sure though, an example would be great.

How about the following?:
Create a custom XMLStreamWriter that will treat all namespace declarations as default namespaces, and then marshal to that:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
XMLOutputFactory xof = XMLOutputFactory.newFactory();
XMLStreamWriter xsw = xof.createXMLStreamWriter(System.out);
xsw = new MyXMLStreamWriter(xsw);
m.marshal(iq, xsw);
xsw.close();
MyXMLStreamWriter
import java.util.Iterator;
import javax.xml.namespace.NamespaceContext;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.XMLStreamWriter;
public class MyXMLStreamWriter implements XMLStreamWriter {
private XMLStreamWriter xsw;
private MyNamespaceContext nc = new MyNamespaceContext();
public MyXMLStreamWriter(XMLStreamWriter xsw) throws Exception {
this.xsw = xsw;
xsw.setNamespaceContext(nc);
}
public void close() throws XMLStreamException {
xsw.close();
}
public void flush() throws XMLStreamException {
xsw.flush();
}
public NamespaceContext getNamespaceContext() {
return xsw.getNamespaceContext();
}
public String getPrefix(String arg0) throws XMLStreamException {
return xsw.getPrefix(arg0);
}
public Object getProperty(String arg0) throws IllegalArgumentException {
return xsw.getProperty(arg0);
}
public void setDefaultNamespace(String arg0) throws XMLStreamException {
xsw.setDefaultNamespace(arg0);
}
public void setNamespaceContext(NamespaceContext arg0) throws XMLStreamException {
}
public void setPrefix(String arg0, String arg1) throws XMLStreamException {
xsw.setPrefix(arg0, arg1);
}
public void writeAttribute(String arg0, String arg1) throws XMLStreamException {
xsw.writeAttribute(arg0, arg1);
}
public void writeAttribute(String arg0, String arg1, String arg2) throws XMLStreamException {
xsw.writeAttribute(arg0, arg1, arg2);
}
public void writeAttribute(String arg0, String arg1, String arg2, String arg3) throws XMLStreamException {
xsw.writeAttribute(arg0, arg1, arg2, arg3);
}
public void writeCData(String arg0) throws XMLStreamException {
xsw.writeCData(arg0);
}
public void writeCharacters(String arg0) throws XMLStreamException {
xsw.writeCharacters(arg0);
}
public void writeCharacters(char[] arg0, int arg1, int arg2) throws XMLStreamException {
xsw.writeCharacters(arg0, arg1, arg2);
}
public void writeComment(String arg0) throws XMLStreamException {
xsw.writeComment(arg0);
}
public void writeDTD(String arg0) throws XMLStreamException {
xsw.writeDTD(arg0);
}
public void writeDefaultNamespace(String arg0) throws XMLStreamException {
xsw.writeDefaultNamespace(arg0);
}
public void writeEmptyElement(String arg0) throws XMLStreamException {
xsw.writeEmptyElement(arg0);
}
public void writeEmptyElement(String arg0, String arg1) throws XMLStreamException {
xsw.writeEmptyElement(arg0, arg1);
}
public void writeEmptyElement(String arg0, String arg1, String arg2) throws XMLStreamException {
xsw.writeEmptyElement(arg0, arg1, arg2);
}
public void writeEndDocument() throws XMLStreamException {
xsw.writeEndDocument();
}
public void writeEndElement() throws XMLStreamException {
xsw.writeEndElement();
}
public void writeEntityRef(String arg0) throws XMLStreamException {
xsw.writeEntityRef(arg0);
}
public void writeNamespace(String arg0, String arg1) throws XMLStreamException {
}
public void writeProcessingInstruction(String arg0) throws XMLStreamException {
xsw.writeProcessingInstruction(arg0);
}
public void writeProcessingInstruction(String arg0, String arg1) throws XMLStreamException {
xsw.writeProcessingInstruction(arg0, arg1);
}
public void writeStartDocument() throws XMLStreamException {
xsw.writeStartDocument();
}
public void writeStartDocument(String arg0) throws XMLStreamException {
xsw.writeStartDocument(arg0);
}
public void writeStartDocument(String arg0, String arg1) throws XMLStreamException {
xsw.writeStartDocument(arg0, arg1);
}
public void writeStartElement(String arg0) throws XMLStreamException {
xsw.writeStartElement(arg0);
}
public void writeStartElement(String arg0, String arg1) throws XMLStreamException {
xsw.writeStartElement(arg0, arg1);
}
public void writeStartElement(String arg0, String arg1, String arg2) throws XMLStreamException {
xsw.writeStartElement("", arg1, arg2);
if(null != arg2 || arg2.length() > 0) {
String currentDefaultNS = nc.getNamespaceURI("");
if(!arg2.equals(currentDefaultNS)) {
writeDefaultNamespace(arg2);
nc.setDefaultNS(arg2);
}
}
}
private static class MyNamespaceContext implements NamespaceContext {
private String defaultNS = "";
public void setDefaultNS(String ns) {
defaultNS = ns;
}
public String getNamespaceURI(String arg0) {
if("".equals(arg0)) {
return defaultNS;
}
return null;
}
public String getPrefix(String arg0) {
return "";
}
public Iterator getPrefixes(String arg0) {
return null;
}
}
}

You can nowadays control prefixes also using a custom mapper.
NamespacePrefixMapper namespacePrefixMapper = new com.sun.xml.bind.marshaller.NamespacePrefixMapper() {
private Map<String, String> prefixes;
{
prefixes = new HashMap<>(3);
prefixes.put(XMLConstants.XML_NS_URI, XMLConstants.XML_NS_PREFIX);
prefixes.put(XMLConstants.W3C_XML_SCHEMA_INSTANCE_NS_URI, "xsi");
prefixes.put(XMLConstants.W3C_XML_SCHEMA_NS_URI, "xs");
prefixes.put(WellKnownNamespace.XML_MIME_URI, "xmime");
}
#Override
public String getPreferredPrefix(String namespaceUri, String suggestion,
boolean requirePrefix) {
String prefix = suggestion == null ? prefixes.get(namespaceUri)
: suggestion;
return prefix == null ? XMLConstants.DEFAULT_NS_PREFIX : prefix;
}
};
marshaller.setProperty("com.sun.xml.bind.namespacePrefixMapper",
namespacePrefixMapper);

How to get tree structure from SVN using java

Is there any way to get the SVN structure as a tree struture in java?
For Eg: if i specify a path http://sample.com/repository/pag/branches/dev/Structure/services/
I want what all entries under services and if it again contains a directory its enteries also in a tree?
Thanks.
Note : i have seen the getDir(). But here i have to keep on iterating it.

If you need all the tree, you may do that with "status" request with report telling that you have an empty working copy. One "status" request should be faster than a number of getDir() requests. An example how to do that with SVNKit
final SVNRepository svnRepository = SVNRepositoryFactory.create(url);
try {
svnRepository.status(revision, "", SVNDepth.INFINITY, new ISVNReporterBaton() {
#Override
public void report(ISVNReporter reporter) throws SVNException {
reporter.setPath("", null, revision, SVNDepth.INFINITY, true);
reporter.finishReport();
}
}, new ISVNEditor() {
#Override
public void targetRevision(long revision) throws SVNException {
}
#Override
public void openRoot(long revision) throws SVNException {
System.out.println("<root>");
}
#Override
public void deleteEntry(String path, long revision) throws SVNException {
}
#Override
public void absentDir(String path) throws SVNException {
}
#Override
public void absentFile(String path) throws SVNException {
}
#Override
public void addDir(String path, String copyFromPath, long copyFromRevision) throws SVNException {
System.out.println("directory: " + path);
}
#Override
public void openDir(String path, long revision) throws SVNException {
}
#Override
public void changeDirProperty(String name, SVNPropertyValue value) throws SVNException {
}
#Override
public void closeDir() throws SVNException {
}
#Override
public void addFile(String path, String copyFromPath, long copyFromRevision) throws SVNException {
System.out.println("file: " + path);
}
#Override
public void openFile(String path, long revision) throws SVNException {
}
#Override
public void changeFileProperty(String path, String propertyName, SVNPropertyValue propertyValue) throws SVNException {
}
#Override
public void closeFile(String path, String textChecksum) throws SVNException {
}
#Override
public SVNCommitInfo closeEdit() throws SVNException {
return null;
}
#Override
public void abortEdit() throws SVNException {
}
#Override
public void applyTextDelta(String path, String baseChecksum) throws SVNException {
}
#Override
public OutputStream textDeltaChunk(String path, SVNDiffWindow diffWindow) throws SVNException {
return null;
}
#Override
public void textDeltaEnd(String path) throws SVNException {
}
});
} finally {
svnRepository.closeSession();
}

get contents of processed JSP into spring controller without using HttpClient?

So normally in a Spring controller you'd just return a ModelAndView object and forward the request to the JSP.
What I need to do is actually get the contents of that processed JSP so I can then send it in a JSONP response (ex: callback("processed HTML from JSP");)
I know I could just use HttpClient to get the contents but was wondering if there's a way to avoid that extra step by calling something like:
String contents = processJSP(modelAndView);
Updated for geek to show my final solution:
First you need a fake HttpResponse to hold the response:
#Service
public class SpringUtils {
private static final Logger LOG = Logger.getLogger(SpringUtils.class);
#Autowired private ViewResolver viewResolver;
#Autowired private LocaleResolver localeResolver;
public String renderView(HttpServletRequest request, ModelAndView mav) {
try {
View view = viewResolver.resolveViewName(mav.getViewName(), localeResolver.resolveLocale(request));
HttpServletResponse localResponse = new MyHttpServletResponseWrapper(new DummyResponse());
view.render(mav.getModel(), request, localResponse);
return localResponse.toString();
} catch (Exception e) {
return "";
}
}
public boolean doesViewExist(HttpServletRequest request, String viewName) {
try {
if (viewResolver.resolveViewName(viewName, localeResolver.resolveLocale(request)) != null) {
return true;
}
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
return false;
}
static class MyHttpServletResponseWrapper extends HttpServletResponseWrapper {
private StringWriter sw = new StringWriter();
public MyHttpServletResponseWrapper(HttpServletResponse response) {
super(response);
}
public PrintWriter getWriter() throws IOException {
return new PrintWriter(sw);
}
public ServletOutputStream getOutputStream() throws IOException {
throw new UnsupportedOperationException();
}
public String toString() {
return sw.toString();
}
}
}
DummyResponse
public class DummyResponse implements HttpServletResponse {
public DummyResponse() {
}
public void setAppCommitted(boolean appCommitted) {}
public boolean isAppCommitted() { return false; }
public int getContentCount() { return -1; }
public boolean getIncluded() { return false; }
public void setIncluded(boolean included) {}
public String getInfo() { return null; }
public ServletResponse getResponse() { return null; }
public OutputStream getStream() { return null; }
public void setStream(OutputStream stream) {}
public void setSuspended(boolean suspended) {}
public boolean isSuspended() { return false; }
public void setError() {}
public boolean isError() { return false; }
public ServletOutputStream createOutputStream() throws IOException {
return null;
}
public void finishResponse() throws IOException {}
public int getContentLength() { return -1; }
public String getContentType() { return null; }
public PrintWriter getReporter() { return null; }
public void recycle() {}
public void write(int b) throws IOException {}
public void write(byte b[]) throws IOException {}
public void write(byte b[], int off, int len) throws IOException {}
public void flushBuffer() throws IOException {}
public int getBufferSize() { return -1; }
public String getCharacterEncoding() { return null; }
public void setCharacterEncoding(String charEncoding) {}
public ServletOutputStream getOutputStream() throws IOException {
return null;
}
public Locale getLocale() { return null; }
public PrintWriter getWriter() throws IOException { return null; }
public boolean isCommitted() { return false; }
public void reset() {}
public void resetBuffer() {}
public void setBufferSize(int size) {}
public void setContentLength(int length) {}
public void setContentType(String type) {}
public void setLocale(Locale locale) {}
public Cookie[] getCookies() { return null; }
public String getHeader(String name) { return null; }
public Collection<String> getHeaders(String arg0) { return null; }
public Collection<String> getHeaderNames() { return null; };
public String[] getHeaderValues(String name) { return null; }
public String getMessage() { return null; }
public int getStatus() { return -1; }
public void reset(int status, String message) {}
public void addCookie(Cookie cookie) {}
public void addDateHeader(String name, long value) {}
public void addHeader(String name, String value) {}
public void addIntHeader(String name, int value) {}
public boolean containsHeader(String name) { return false; }
public String encodeRedirectURL(String url) { return null; }
public String encodeRedirectUrl(String url) { return null; }
public String encodeURL(String url) { return null; }
public String encodeUrl(String url) { return null; }
public void sendAcknowledgement() throws IOException {}
public void sendError(int status) throws IOException {}
public void sendError(int status, String message) throws IOException {}
public void sendRedirect(String location) throws IOException {}
public void setDateHeader(String name, long value) {}
public void setHeader(String name, String value) {}
public void setIntHeader(String name, int value) {}
public void setStatus(int status) {}
public void setStatus(int status, String message) {}
}

Probably not possible easy way.
Maybe injecting view resolver into controller and calling render with special response will help, but not sure :
ViewResolver viewResoler = // injected
View view = viewReslover.resolveViewName(String viewName, Locale locale);
HttpServletResponse xresponse = // custom response, buffers data
view.render(Map model, HttpServletRequest request, HttpServletResponse xresponse);
String content = // extract conten from data from xresponse
'