How should I handle LIST command in my FTP server? - java

I'm writing an FTP server with Java, and now I want to answer to LIST command.Sending only file names is enough, and I don't need to send file size, owner, permission, etc. It seems that just sending some strings, as file names, does not satisfy the client (I tried both ASCII and binary formats). How can I find out what does an FTP client expect as a reply?
I'm testing my server using FireFTP and FileZilla

If you want to create a compatible FTP server, you need to handle LIST and NLST (standard commands) and also MLST and MLSD extension commands.
Format for LIST command is not defined anywhere and there are about 400 formats encountered in the world. Using Unix ls format or Windows DIR format would work with most clients as these are formats quite widespread and well supported by the clients.
NLST is the list of file names only.
MLST and MLSD use the machine-parseable format (this is what M letter stands for) which is described in RFC 3659. It's easier for the clients to handle and it's support is very welcome.

The canonical place to look is the relevant RFC: http://www.ietf.org/rfc/rfc959.txt
Unfortunately, in this particular instance the RFC is pretty vague:
Since the information on a file may vary widely from system
to system, this information may be hard to use automatically
in a program, but may be quite useful to a human user.
In order to ensure compatibility with existing FTP clients, your best bet is to look at some widely-deployed FTP server software and emulate the format of its output.

Related

Implemeting a basic FTP client in Java

I want to write a FTP-Client in Java with a restriction: No advanced libraries (e.g. .ftp, .url etc.) allowed.
How do I implement a method to print the current directory, change directory and download a simple .txt-file?
You can start by reading up the RFC governing the FTP protocol. With that you can get an idea on how the FTP protocol works, how it sends commands, expected responses etc.
You can find a link here: https://www.rfc-editor.org/rfc/rfc959
Aside from that you can have a look at this GitHub repository. In there you'll find a simple FTP client that I wrote back when I was in uni. The FtpConnection class implements most of the commands you'll need to do your job.
Have a look at it and how these are used.
https://github.com/Kortex/Simple-FTP-Client

Stitching DHT and Torrent Client together

I have two java libraries.
One connects to DHT and can successfully download torrent files.
And the other which downloads the actual data associated with the torrent.
The torrent client library expects an announce or announce-list section in the torrent, and does't understand the 'nodes' entry.
The question is how do I alter the torrent client code to understand the 'nodes' section of a torrent file.
OR
How do I calculate a tracker URL from a set of ip:port DHT peer addresses?
I could just guess the url by guessing a port number and appending /announce onto it.. but this surely isn't right?
Does anyone know how this works?
For DHT-only operation you do not need an announce URL. If the library itself expects one you can insert a dummy url, the format dht://<infohash in hex> is common, but it does not matter, as it won't be used.
The DHT part on the other hand does not require the nodes to perform a lookup, it simply operates based on the infohash of the torrent. The nodes can optionally be injected into a dht client for bootstrapping, e.g. by executing a ping on them, but that should not be needed if it already is bootstrapped.
Once the DHT client has done its get_peers and announce lookups the peer lists can be injected back into the torrent client.
Since you're using my library: You can use PeerLookupTask to read a torrent peer list from the DHT and AnnounceTask to add your torrent port to the list. The obtained IP and Ports have to be passed to the torrent client. The GetPeers CLI command contains an example how to use the first part.
There also are utility methods that should cover common use cases.
For an alternative I may recommend you to see Bt, which is a Java 8 BitTorrent client and already integrates with the8472's mldht: https://github.com/atomashpolskiy/bt

Email intercepting techniques

I have a requirement where I intend to build a tool to scan the email contents including the attachments. The email servers is either going to be SendMail or z/OS Communication Server, both support SMTP. The sever is not Miscrosoft implementation so MAPI or Outlook API is not there into picture. The tool would be Java based code and basically need to look for contents that are not-permitted based on some rules. What are my options here? There is the possibility of using a proxy server but we are looking for a more direct approach.
The z/OS Communication Server SMTP implementation has a built-in "exit" capability - see http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/F1A1B4B0/30.3?DT=20110609204120#HDRWQ1299.
The exit is called for just about any SMTP activity and it can examine, change or reject just about anything based on the rules you establish. It is generally written in IBM Assembler Language, but there's no reason you couldn't have a thin assembler layer that passes data to a Java app using whatever protocol you like (say, a pipe or a socket).
There are many little details to handle, such as character encoding (EBCDIC vs. ASCII or UTF-8, for example) plus weeding out attachments from email content. But using the exit preserves all the z/OS specific features of IBM's SMPT server without trying to recreate any of that yourself.
Good luck!

& becomes & during FTP to MVS

I am using a java library (edtftpj) to FTP a file from a web app hosted of a tomcat server to an MVS system.
The FTP transfer mode is ASCII and transfer is done using FTP streams. The data from a String variable is stored into an MVS dataset.
The problem is all the ampersand characters get converted to & . I have tried various escape characters including \& , ^& and X'50' (hex value), but none of it helped.
Anyone has any idea how to escape the ampersands please?
Nothing in the FTP protocol would cause this encoding behavior.
Representing & as & is an XML based escaping representation. Other systems might use the same scheme, but as a standard, this is an XML standard encoding.
Something in the reading of the data and writing of the data thinks it should be escaping this information and is doing the encoding.
If anything on the MVS system is using Java it is probably communicating via SOAP with some other connector, which implies XML, which could be causing the escape sequence to be happening.
Either way, the FTP protocol itself part is not part of the problem, ASCII transfer should only encode things like line endings, & is already a valid ASCII character and would not be affected. It is the MVS system that is doing this escaping if anything.
Binary transfer is preferred in almost every case, since it doesn't do any interpretation or encoding of the raw bytes.
Using FTP in ASCII-mode to/from a MVS (z/OS) will always perform code page conversions (i.e ASCII <-> EBCDIC) for the data connection. Thus it's very important to setup the connection with the appropriate parameters depending on dataset type and page codes. Example:
site SBD=(IBM-037,ISO8859-1)
site TRAck
site RECfm=FB
site LRECL=80
site PRImary=5
site SECondary=5
site BLKsize=6233
site Directory=50
As alternative, use BINARY mode and manually perform the conversions with some of the standard tools or libraries on the receiving end.
Ref links:
1. Preset commands to tackle codepage problem.
2. Coverting ASCII to EBCDIC via FTP on MVS Host.
3. Transferring Files to and from MVS.
4. FTP code page conversion.
5. FTP File Transfer Protocol and Z/OS (pdf).

Is there a library for reading /var/spool/mail/ files?

How does one parse the mail files generated on a Linux system using Java? I mean, I want to be able to extract the From, To, Timestamp and Subject of the various emails inside the file. Any suggestions?
javax.mail.internet.MimeMessage.parse(InputStream)
it's protected but you can subclass it to use the method. However, the file format is quite simple, if you just want some headers, why not parse it on your own?
Those files belong to the Mail Transfer Agent and maybe the user's mail client. Other programs should tread very softly or better yet keep out altogether. Or is your program a mail client?
The "clean" way to do this would be to open up an SMTP or IMAP connection to the mail server / MTA and ask it for pieces of mail on behalf of your user, using his credentials that he gives you.
There's a Java mail API for this that knows how to do this well: http://java.sun.com/products/javamail/ .

Categories