I'm using Solarflare cards with sfptpd and I'm consuming the hardware timestamps in my C code. It is not much different from the Solarflare example that gives the following output:
$ export EF_RX_TIMESTAMPING=3
$ onload ./rx_timestamping --proto udp --port 6666
oo:rx_timestamping[32356]: Using OpenOnload 201509-u1 Copyright 2006-2016 Solarflare Communications, 2002-2005 Level 5 Networks [1]
Socket created, listening on port 6666
Socket accepted
Selecting hardware timestamping mode.
Packet 1 - 3 bytes timestamps 1468588400.706881929 1468588400.706878923 1468588400.706878923
Packet 2 - 3 bytes timestamps 1468588403.815893129 1468588403.815890313 1468588403.815890313
The idea is to inspect the ancillary data associated with the msghdr structure and extract the timespec from the SO_TIMESTAMPNS (well, it would be almost the same if I wanted to rely on software kernel timestamps - without Solarflare, onload, etc).
So far so good. I would like to know if there is any way to access these timestamps using the standard Java APIs? If not, then what would be the easiest way to proceed? I don't mind hacking some JNI code. In the worst case I could handle networking in the native code providing Java with the received data along the metadata (but if possible I would like to avoid doing that as I'm looking at accessing the timestamps from an existing app).
Related
All,
Not sure whether to create some Java code or use an 'off-the-shelf' product to produce a listener / router / redirector, one which will listen to a single incoming IP stream and then 'redirect' packets based on a fixed position string variable to an appropriate TCP server.
i.e. If the incoming packet has variable = 1 redirect to a TCP server at 192.168.150.211, if a 2 then redirect to TCP server at 192.168.150.212.
Ideally the code / product should also be able to pre-launch multiple TCP servers (either at one IP address and different IP ports or at different IP addresses) on the same machine prior to the listener / router starting.
Was thinking of using an OPC server but these seem too complex / costly for a single, relatively slow, incoming stream.
Thoughts appreciated.
This depends on the depth of analyzing. If you would like to analyze each packet at TCP-level then low-level language like C will be the best choice: just listen everything that comes to eth0 (or whatever interface), search for specific string (BTW "string" definition is too wide. The times when a string could be considered as ready-to-process data are in the '80s. Nowadays string is a piece of rubbish until you know what encoding is, how lines are terminated, etc).
As already mentioned by #f1sh you can redirect traffic using Java. But Java operates with streams, it knowns nothing about network packets. Also, in Java you can listen only on certain port(s) - there is no way to filter on the whole network interface (but JNI can be real saver).
If you would like to get PoC without huge amount of coding lets consider Socat. Socat can not only transmit data from one socket to another. It can also write data to files. So you can combine Socat (don't forget about fork option), tail, grep and a bit of Bash stuff and get simple redirection server working.
The protocol IEC 62056:21 tells us how to deal with enegy meters, it's quite easy!
The part where I am stuck is the implementation over a GSM data channel. Normally I would set things like:
300 baudrate
1 parity bit (even)
But the meter is not connected via serial connection but, instead, it has a sim. Using a modem I can call the meter using:
AT&C1
ATDNumber
Problem 1: Settings
The modem calls the meter with different settings (baudrates, stopbits, parity) compared to the protocol ones, e.g.
9600 baudrate for call
300 baudrate for first messages
xxxxx new baudrate shared between master and slave
Can I change these parameters during call?
Problem 2: Send data
After I establish a call, I would send to the meter things like:
/ ? Device address ! CR LF
Here's the missing piece, I don't know how to send this data over the call
I am reading and trying several libraries (like J62056, pyserial), but I've found nothing about sending data via gsm call
EDIT
I read a trace of a proprietary software, and I got this:
TX: 140ms AT&C1E0V0
RX: 32ms 0
TX: 1203ms
ATDT ##########
RX: 34656ms 1
RX: 0ms 5
RX: 0ms
TX: 3234ms <NUL><NUL><NUL><NUL><NUL><NUL><NUL><NUL> *what is this?*
TX: 594ms /?########! (this the Request message) **start sending data**
The < NUL > part is not Clear, and this is where the modem starts to send data
Edit:
I read about the 8 null chars, they're just a check-in sequence.
At the moment, after the modem established the call I translate my 8 bit no parity sequence into a 7 + parity one. Now i am able to send and receive data from the meter, I must test other feature before writing my solution to this answer
Without knowing anything about IEC 62056:21, if your energy meter supports this over GSM circuit switched data (CSD), nothing it says about speed and parity in the normal non-GSM case is relevant at all.
Because the data call you set up will be non-transparent CSD (NTCSD). A transparent call would have treated the gsm connection as much as just an electrical wire as possible (which in best case is difficult1), forwarding each byte received immediately and with no buffering nor with any retransmission support. A non-transparent connection on the other hand will receive/send its data to an intermediate entity which in turn communicates with the other end point, and will support buffering and retransmission.
For GSM NTCSD, the part of the phone responsible for data handling is called TAE (Terminal Adapter Equipment) or TAF (T. A. function), and the relevant protocol2 is called RLP (Radio Link Protocol) which is specified by 3GPP in specification 24.022. It is a link-layer protocol similar to HDLC, and it communicates with a unit in the GSM network called MSC (Message Switching Centre). It is then then MSC which communicates with the other end, on a different and completely separate communication line (which can be PSTN, ISDN or mobile network depending on what kind of device the remote end is).
mobile modem <----link1----> MSC <----link2----> remote endpoint
The important thing here is that the two links are 100% independent, and they do not have to be the same speed.
Thus whatever speed your energy meter is using over some serial interface between itself and the embedded modem with SIM card is independent of the radio link1 speed which itself is independent of the network link2 speed which your modem will have.
So the above should be an answer to your question, but let me fill in some more information with regards to the speeds of link1 and link2, because this can be controlled with two AT commands AT+CBST and AT+CHSN (specified in 27.007).
The basic GSM data traffic channel is called TCH/9.6 (Traffic Channel), which is a channel with net 9600bit/s speed (seen by the user) and 12000bit/s gross speed (seen by the network). In order to enhance throughput
HSCSD (High Speed CSD) was developed which introduced a new channel coding and a new channel TCH/14.4 which had 14500bit/s gross speed and 13200bit/s net speed (which of course all the marketing people presented as 14.4 speed even though that not was really true).
In addition HSCSD allowed for bundling multiple timeslots together (multislot). A frequency band in GSM is divided into 8 timeslots, where an active call occupies one timeslot in the downlink direction and one timeslot in the uplink direction. Thus a cell tower configured to support only one frequency band supports maximum 8 simultaneous calls.
What HSCSD introduced was the possibility to set up a call that could use multiple (adjacent) timeslots, for instance two downlink timeslots and one uplink (denoted 2+1). The different multislot configurations a phone or network supported were categorised in multislot classes3 (with 2+1 being multislot class 10 as far as I remember).
Since HSCSD both was an added value and occupied more network resources this was something that was billed higher than a normal 1+1 9600 call, and thus the users had to have some control over if they used HSCSD or not. This was
done by introducing the AT+CHSN command which controls the link1 speed.
Support for hinting to the MSC what speed it should use for link2 was implemented by the AT+CBST command (which already existed before HSCSD).
Since the AT command is terminated in the mobile phone, its value had to be forwarded to the MSC in some way, and this was done out-of-band (with regards to the RLP data link) in a Bearer Capability Information Element in some of the call setup messages.
So, that's probably more than you need to know about speeds in a GSM network for a NTCSD call, but having developed and maintained the NTCSD call stack in Ericsson's mobile phones for over a decade, this is something I know quite well...
1
While there are standardized some support for transparent data, this was more of a legacy thing done in the 90-s in order to support equipment made in the 80-s. Operators do not want to have this supported today because it is a pain in the butt to get working and to support.
Fax over GSM for instance was such a transparent bearer, and it was a massive test everywhere approach with no guarantee that it would work even if you followed the specification fully. You had to do your best effort implementation and then you had to travel all over the world testing it and try to fix all the issues that would pop up (which they absolutely did. And even if the problem was something wrong with the network, the operator might not want to fix it so you had to add some custom workaround).
One of the guys working with fax support told me that in some cases they had to start sending the response before they had gotten the request in order for the timing to work out (e.g. they had to guess and anticipate what the remote fax would do).
It is not without a reason that phones manufactured and operators today do not support fax like they did in the 90-s, early 2000.
2
In addition there is L2RCOP which is just a framing adoption between packet based RLP and the physical serial interface.
3
CSD and GPRS both supported multislot, but not necessarily the same class, i.e. CSD multislot class is independent from GPRS multislot class.
I am using this library for communicating with serial ports from Java. Also I am using USB to Serial converter to connect to the device.
Here is the documentation that is related to the device :
2.1 Physical Interface
The required physical interface between the host and the VGM is the EIA-232 interface.
2.2 Logical Interface
The serial data link shall operate at the speed of 19,200 bits per second (BPS), with one
start bit, eight data bits, a wake-up bit and one stop bit. The wake-up bit should be set in
the first byte of the message; the wake-up bit should be cleared for the remainder of the
message. The VGM shall clear the wake-up bit when responding to the host.
I am a little puzzled how to setup the rs232 library settings when connecting to the serial port. There are baud settings, data length in bits (5, 6, 7, 8), stop bits (1, 2) and parity setting. When I mess with these settings I of course get different output (most of the time looking like trash). Can you help me guess the settings with regards to the quoted documentation?
From what I understand your protocol requires an additional 9th data bit which is used in some exotic applications like Multidrop bus (see also Stackoverflow 14212660). In your case this 9th bit is called "wake-up bit", but you will not find such a thing or name in your java library or and standard RS232 application.
There is a workaround using standard USB to Serial converters. It is exactly what in Stackoverflow 14212660 is called
and no fudging by using the parity bit as a 9th data bit
So, unless you want to buy specialized hardware - I suggest the 'fudging':
Using the parity settings MARK and SPACE should correspond to your desired setting "wake up bit set", resp. "wake up bit cleared". Our software Docklight Scripting allows you in the free evaluation already to do this kind of temporary parity switching, but I assume there are also other tools or code examples around. MDB / multidrop bus should be good Google keywords for this.
I am trying to understand the CANopen protocol.
For now, I do not have any CAN hardware nor the CANopen stack to experiment with.
I would like to know how to write a Java program to simply interpret CANopen messages that are received at the RS-232 port.
Are there CAN interfaces that are installed as a serial port?
Will I be able to write a program to process CANopen messages? I want only to be able to receive and interpret messages. Is it as simple as creating a buffer for the input stream and then break up the transmission into separate messages according to the SOF and EOF? How do I know what is the SOF/EOF since it is only 1-bit long?
Why is there a limit on the number of PDOs of a CAN node?
How do I process the PDO to identify the node from which it is sent and the data type and value? Is the PDO a standard CAN frame?
I don't know of any CAN interface that connects to the serial port (it wouldn't be too hard to create one based on a microcontroller with CAN and serial ports). However, standard serial ports would be too slow to support the higher speeds available in CAN.
Generally, when using the API for a CAN interface, you will be able to read messages consisting of ID, Length and up to eight bytes of data. You don't need to care about SOF/EOF. Even if interfacing directly on the low level with a CAN controller (that is, if you have a CAN interface for which you need to write the driver/API yourself), you still don't need to care about those details. And you don't want to try to access the CAN bus without using a CAN controller at all...
If you want to pretend that you have a CAN interface, you may create a stub function which returns those three items: an ID, a data length and a 64-bit data buffer. This is basically what all CAN interface APIs will give you. And when transmitting CAN messages, you will use the same parameters (ID, length data).
PDOs are defined by their use of the CAN ID field. In theory the number of PDOs for a device is not really that limited, but the predefined connection set have only allocated a small number (four) of PDOs for each node.
The PDOs are standard CAN frames. As mentioned, the CAN ID identifies the PDO. In the predefined connection set (which most devices follow), the CAN ID of all messages consists of a functional part and a module-ID part (the module ID may be hard coded for the device, or configurable by dip switches for example). Bits 10-7 of the CAN ID is the function code and bit 6-0 is the module number. For example TxPDO1 from a device with module ID 0x10 would have CAN ID 0x190. The upper four bits of the 11-bit CAN ID, ((CAN_ID & 0x780) >> 7), gives you the function code (TxPDO1 = 3) and the rest of the bits,(CAN_ID & 0x7f), gives the module id (which in this example was 0x10). So if you read a message on the CAN bus with CAN ID 0x190, you would know that this was a PDO from the device with module ID 0x10.
(A simpler way to express this might be to say that TxPDO1 has CAN ID set to 0x180+<module ID>, TxPDO2 has CAN ID set to 0x280+<module ID>, etc.)
How you should interpret the data in the PDO depends on your device.
I suggest that you find a good CANopen tutorial. Unfortunately most of them make everything sound much more complicated than it really is. So look around until you find one that appears understandable.
There are many CAN interfaces that can run off a serial port - VSCOM, Vector, and many others. There are also free programs that allow you to send and receive raw CAN frames - CANhacker, etc. Google for a few of them.
What I haven't found is a free program that can do interpret CANopen - most are pay programs. The exception is Wireshark for Linux - it uses SocketCAN to pull in packets and can parse all the CANopen frames.
I run my CAN bus a 1 Mbit/s and use a VSCOM interface to monitor it on the serial port.
CANFestival is a good open source stack that ports easily to Linux as well as bare machines.
I would like to discover the type of the NAT (FullCone, Restricted Cone, Port Restricted cone, Symmetric) a given network interface is behind.
I've tested different tools (http://freshmeat.net/projects/jstun/, http://code.google.com/p/boogu/) but they report different results for the same interface.
I'm looking for a definitive answer in Python (or other languages, 2nd choice being Java, if nothing else is available).
I second the answer from #S.Lott: It is not possible to use STUN (or any other protocol) to determine with 100% certainty what type of NAT you're behind.
The problem is (as I witnessed recently) that the NAT may sometimes act as Address-Dependent (Symmetric) and sometimes as Endpoint-Independent (Full, Restricted or Port Restricted cone).
When you think about it, being Address-Dependent means that when you send packets from one socket on a client behind the NAT to two distinct servers, then the NAT will create two custom public address:port tuples for each of the servers. In my case, these bindings seemed completely random, but if the range is small, it sometimes happened that these tuples were actually equal! Which confused the test.
I was using this library at the time and sometimes it told me the NAT's behavior was Address-Dependent and other times it was Endpoint-Independent (the switch between the two also seemed completely random, sometimes it happened after I restarted the device, sometimes after a time period,...).
This happened to me on a mobile device with Slovak Telekom, a company mostly owned by Deutsche Telekom so I think the problem will be at least Europe-wide.
I would say that the rule here is this: If STUN test tells you that you're behind a Symmetric NAT than that is the case, but if it tells you otherwise, then you can not be 100% sure.
One last note, an easy way to check your NAT's behavior with respect to TCP is to type "what is my IP address" into google and then open first (say) five pages. If the pages are not consistent about your IP address, your NAT's behavior is Address-Dependent or Address-and-Port-Dependent (Symmetric). But again, if they do correspond, you just can't be sure.
http://en.wikipedia.org/wiki/STUN
NAT devices are implemented in a
number of different types of address
and port mapping schemes. STUN does
not work correctly with all of them.
Is that definitive enough? It's only a wikipedia quote, but from this it looks like your request is physically impossible.
As #S.Lott 's say, STUN is your first choice protocol .
And then, STUN is just a protocol.Here is my advice:
1 STUN now has two version : the old version is RFC3489 - this is a lightweight protocol that allows applications to discover the presence and types of NATs and firewalls between them and the public Internet (so it's mainly and only for detecting NAT type ); and the new version is RFC5389 - this is a tool for other protocols in dealing with NAT traversal .
2 Also there is a relay extension to STUN named TURN RFC5766. TURN allows the host to control the operation of the relay and to exchange packets with its peers using the relay. TURN differs from some other relay control protocols in that it allows a client to communicate with multiple peers using a single relay address.
The tools:
STUN server (RFC3489) : stund By c++
STUN client (RFC3489) : pystun By python
TURN server (RFC5766) : turnserver By c
TURN client (RFC5766) : turn-client By c and python
Note:
Because TURN is the extension of new version STUN, the TURN server also support new STUN request by RFC5389 .