We Keep Coding

Java download causes lag spike with really small files

EDIT: I've decided to remove the Minecraft tag. While this IS a Minecraft mod's code, the code I'm having trouble with has to do with functions within core Java, and not anything in Minecraft itself.
So, I posted a question last night about me writing a function of a Minecraft mod to allow us to implement our own capes and thanks to some of you I was able to get that working. I've crammed down the code from there and found a way to make it as small as possible while producing the same outcome. I've also found out how to read files directly from a download stream instead of copying the file. Great? Well, no. Now that I'm able to properly test everything and read my files, I've run into another problem. While everything works fine, it causes a lag spike of upwards to 2-4 seconds whenever my code is running. I've looked around and found issues about slow downloads, but nothing about the application completely freezing for a small period of time. Since this is run whenever a player first appears on-screen, this can get repetitive. I'm trying to fix the code and have rearranged it countless times in different ways and it just won't go away.
Here's the code. In short, it checks if the player has an official Mojang cape on their account. If they don't, then instead it runs my code to check if they have one of our custom capes, and exactly when this is run, is when the game has a lag spike. Note that if the player has a cape from Mojang my code is never run and I've tried disabling my code and just running the code that downloads their Mojang cape for use, so I know that part doesn't cause lag, it's specifically just my code. Even if they don't have a custom cape under my code it always causes lag. I can't think of why; the files are very small!
... While writing this, I decided to test something, and so I disabled the player.cloakUrl variable setting lines in my getCustomCapes method. Surely enough the lag still happened so I reckon the lag is caused by the lines that check if the player UUID matches one in the downloaded .txt files. I'm confident the lines of code that check and download from the .txt files are the culprits to the lag here. Note that I'm reading from streamed online files rather than saving them to the computer permanently since they don't need to. They're read every time this code is called and it needs to stay that way.
Without further ado here's the code.
EDIT: I've tried to add .close() commands to the streams with no success.
public static void getSkin(final GameProfile profile, EntityPlayer player) {
final Map<Type, MinecraftProfileTexture> map = Maps.<Type, MinecraftProfileTexture>newHashMap();
// (Here's where some skin-related code would be but I cut it out for easier reading,
// it has nothing to do with this situation so don't worry about that.)
if (map.containsKey(Type.CAPE))
{
MinecraftProfileTexture texture = (MinecraftProfileTexture)map.get(Type.CAPE);
player.cloakUrl = texture.getUrl();
return;
} else {
getCustomCapes(profile, player); //Commenting this fixes the lag, proving it's my code that causes it.
}
}
public static void getCustomCapes(final GameProfile profile, EntityPlayer player) {
InputStream staff;
InputStream contest;
try {
contest = new URL("https://www.dropbox.com/s/kvvkfk6emms3qg5/contest.txt?dl=1").openStream();
StringWriter contestUUIDs = new StringWriter();
IOUtils.copy(contest, contestUUIDs, StandardCharsets.UTF_8);
if (contestUUIDs.toString().contains(profile.getId().toString()))
{
player.cloakUrl = "https://www.dropbox.com/s/a4d3agty57sd4a8/contest.png?dl=1";
// Commenting out the above line doesn't seem to help any, as stated before.
return;
}
staff = new URL("https://www.dropbox.com/s/q6f4729i2zu02nz/staff.txt?dl=1").openStream();
StringWriter staffUUIDs = new StringWriter();
IOUtils.copy(staff, staffUUIDs, StandardCharsets.UTF_8);
if (staffUUIDs.toString().contains(profile.getId().toString()))
{
player.cloakUrl = "https://www.dropbox.com/s/42jn5r7fs6k5f4l/staff.png?dl=1";
// Commenting out the above line doesn't seem to help any, as stated before.
return;
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}

What an old thread, huh? So, after getting a little more experience I decided to head back to this to say I've found the solution. Here's how I fixed it: Thread magic! ug_ suggested this to me a while ago it seems, but I simply didn't understand at the time. I used some methods to start a thread with special variables that set the player's properties when it's done, removing the lag. Also, I rewrote the way the cape .txts work to be a little more efficient. You may notice there's one file now, and that one controls everything instead of each list having its own file. Here's my code. First, the above get skin methods have been scraped out almost entirely, and I added a new one since creating a GameProfile sends requests to Mojang's API, causing a little bit of lag, so I made it so this thread can also generate profiles:
public static void getSkin(final GameProfile profile, EntityPlayer player) {
ThreadFetchSkins skinThread = new ThreadFetchSkins();
skinThread.start(player, profile, mc);
}
public static void getSkinFromUUIDAndName(UUID uuid, String username, EntityPlayer player) {
ThreadFetchSkins skinThread = new ThreadFetchSkins();
skinThread.start(uuid, username, player, mc);
}
And then, here's the thread.
public class ThreadFetchSkins extends Thread {
public static TreeMap<UUID, String> customCapes = new TreeMap<UUID, String>();
private EntityPlayer thePlayer;
private GameProfile theProfile;
private Minecraft mc;
private boolean createProfile = false;
private UUID id;
private String name;
public ThreadFetchSkins() {
setName("Cape data download thread");
setDaemon(true);
}
public void run() {
if(createProfile) {
theProfile = new GameProfile(id, name);
}
final Map<Type, MinecraftProfileTexture> map = Maps.<Type, MinecraftProfileTexture>newHashMap();
theProfile.getProperties().clear();
theProfile.getProperties().putAll(SkinManager.getProfileProperties(theProfile));
map.putAll(mc.sessionService.getTextures(theProfile, false));
//Removed "secure" textures grab
if (map.containsKey(Type.SKIN))
{
MinecraftProfileTexture texture = (MinecraftProfileTexture)map.get(Type.SKIN);
thePlayer.skinUrl = texture.getUrl();
String grab = texture.getMetadata("model");
// Currently returns null for steve skin
thePlayer.modelAlex = !(grab == null);
thePlayer.worldObj.obtainEntitySkin(thePlayer);
}
if (map.containsKey(Type.CAPE))
{
MinecraftProfileTexture texture = (MinecraftProfileTexture)map.get(Type.CAPE);
thePlayer.cloakUrl = texture.getUrl();
thePlayer.worldObj.obtainEntitySkin(thePlayer);
}
else {
grabCustomCapes();
}
}
public void grabCustomCapes() {
try {
URL capeDownload = new URL("https://www.dropbox.com/s/q6f4729i2zu02nz/capes.txt?dl=1");
String capeFolder = Minecraft.getMinecraftDir().getPath() + "/NFCCapes";
FileUtils.forceMkdir(new File(capeFolder));
FileUtils.copyURLToFile(capeDownload, new File(capeFolder + "/capes.txt"));
InputStream fileOS = new FileInputStream(capeFolder + "/capes.txt");
BufferedReader buf = new BufferedReader(new InputStreamReader(fileOS));
String line = buf.readLine();
customCapes.clear();
while(line != null) {
String[] data = line.split(";");
customCapes.put(UUID.fromString(data[1]), data[2]);
line = buf.readLine();
}
fileOS.close();
FileUtils.getFile(capeFolder).delete();
if(customCapes.get(theProfile.getId()) != null) {
thePlayer.cloakUrl = customCapes.get(theProfile.getId());
thePlayer.worldObj.obtainEntitySkin(thePlayer);
}
} catch (MalformedURLException e) {
e.printStackTrace();
return;
} catch (IOException e) {
e.printStackTrace();
return;
}
}
public void start(EntityPlayer player, final GameProfile profile, Minecraft minecraft) {
thePlayer = player;
theProfile = profile;
mc = minecraft;
super.start();
}
public void start(UUID uuid, String username, EntityPlayer player, Minecraft minecraft) {
id = uuid;
name = username;
thePlayer = player;
mc = minecraft;
createProfile = true;
super.start();
}
}

Buffer and flush Apache Beam streaming data

I have a streaming job that with initial run will have to process large amount of data. One of DoFn calls remote service that supports batch requests, so when working with bounded collections I use following approach:
private static final class Function extends DoFn<String, Void> implements Serializable {
private static final long serialVersionUID = 2417984990958377700L;
private static final int LIMIT = 500;
private transient Queue<String> buffered;
#StartBundle
public void startBundle(Context context) throws Exception {
buffered = new LinkedList<>();
}
#ProcessElement
public void processElement(ProcessContext context) throws Exception {
buffered.add(context.element());
if (buffered.size() > LIMIT) {
flush();
}
}
#FinishBundle
public void finishBundle(Context c) throws Exception {
// process remaining
flush();
}
private void flush() {
// build batch request
while (!buffered.isEmpty()) {
buffered.poll();
// do something
}
}
}
Is there a way to window data so the same approach can be used on unbounded collections?
I've tried following:
pipeline
.apply("Read", Read.from(source))
.apply(WithTimestamps.of(input -> Instant.now()))
.apply(Window.into(FixedWindows.of(Duration.standardMinutes(2L))))
.apply("Process", ParDo.of(new Function()));
but startBundle and finishBundle are called for every element. Is there a chance to have something like with RxJava (2 minute windows or 100 element bundles):
source
.toFlowable(BackpressureStrategy.LATEST)
.buffer(2, TimeUnit.MINUTES, 100)

This is a quintessential use case for the new feature of per-key-and-windows state and timers.
State is described in a Beam blog post, while for timers you'll have to rely on the Javadoc. Nevermind what the javadoc says about runners supporting them, the true status is found in Beam's capability matrix.
The pattern is very much like what you have written, but state allows it to work with windows and also across bundles, since they may be very small in streaming. Since state must be partitioned somehow to maintain parallelism, you'll need to add some sort of key. Currently there is no automatic sharding for this.
private static final class Function extends DoFn<KV<Key, String>, Void> implements Serializable {
private static final long serialVersionUID = 2417984990958377700L;
private static final int LIMIT = 500;
#StateId("bufferedSize")
private final StateSpec<Object, ValueState<Integer>> bufferedSizeSpec =
StateSpecs.value(VarIntCoder.of());
#StateId("buffered")
private final StateSpec<Object, BagState<String>> bufferedSpec =
StateSpecs.bag(StringUtf8Coder.of());
#TimerId("expiry")
private final TimerSpec expirySpec = TimerSpecs.timer(TimeDomain.EVENT_TIME);
#ProcessElement
public void processElement(
ProcessContext context,
BoundedWindow window,
#StateId("bufferedSize") ValueState<Integer> bufferedSizeState,
#StateId("buffered") BagState<String> bufferedState,
#TimerId("expiry") Timer expiryTimer) {
int size = firstNonNull(bufferedSizeState.read(), 0);
bufferedState.add(context.element().getValue());
size += 1;
bufferedSizeState.write(size);
expiryTimer.set(window.maxTimestamp().plus(allowedLateness));
if (size > LIMIT) {
flush(context, bufferedState, bufferedSizeState);
}
}
#OnTimer("expiry")
public void onExpiry(
OnTimerContext context,
#StateId("bufferedSize") ValueState<Integer> bufferedSizeState,
#StateId("buffered") BagState<String> bufferedState) {
flush(context, bufferedState, bufferedSizeState);
}
private void flush(
WindowedContext context,
BagState<String> bufferedState,
ValueState<Integer> bufferedSizeState) {
Iterable<String> buffered = bufferedState.read();
// build batch request from buffered
...
// clear things
bufferedState.clear();
bufferedSizeState.clear();
}
}
Taking a few notes here:
State replaces your DoFn's instance variables, since
instance variables have no cohesion across windows.
The buffer and the size are just initialized as needed instead
of #StartBundle.
The BagState supports "blind" writes, so there doesn't need to be
any read-modify-write, just committing the new elements in the same
way as when you output.
Setting a timer repeatedly for the same time is just fine;
it should mostly be a noop.
#OnTimer("expiry") takes the place of #FinishBundle, since
finishing a bundle is not a per-window thing but an artifact of
how a runner executes your pipeline.
All that said, if you are writing to an external system, perhaps you would want to reify the windows and re-window into the global window before just doing writes where the manner of your write depends on the window, since "the external world is globally windowed".

The documentation for apache beam 0.6.0 says that StateId is "Not currently supported by any runner."

How to set a single entity invisible to a player?

After much research and much time wasted, I still can't find out how to hide an entity to a player.
What I'm trying to do is create a disguise command. I've now gotten everything worked out, except the issue is that the entity is still visible, and once stationary you can't interact with anything because the mob's hitbox is in the way. I want to hide the entity from the player so that you can do this. I know with players you can use Player#hidePlayer(), but this does not work with entities. I've tried using solutions such as this, but it gave an error while following the example. (And many things were depreciated, so I assumed it was out of date. I'm using Spigot 1.11.2). Any help would be very much appreciated.
PS: If you're wondering why I don't just use an already made plugin, it's because none of them work from what I've found.

To accomplish what you want, you must use packets to cancel what the player sees.
I strongly recommend ProtocolLib, have it in your server and use in your plugin.
Bearing that in mind, Bukkit user Comphenix has developed a class for protocollib to hide entities. You can find it in github.
Comphenix also provides an example of usage, as you can see below:
public class ExampleMod extends JavaPlugin {
private EntityHider entityHider;
private static final int TICKS_PER_SECOND = 20;
#Override
public void onEnable() {
entityHider = new EntityHider(this, Policy.BLACKLIST);
}
#Override
public boolean onCommand(CommandSender sender, Command command, String label, String[] args) {
if (sender instanceof Player) {
final Player player = (Player) sender;
final Sheep sheep = player.getWorld().spawn(player.getLocation(), Sheep.class);
// Show a particular entity
entityHider.toggleEntity(player, sheep);
getServer().getScheduler().scheduleSyncDelayedTask(this, new Runnable() {
#Override
public void run() {
entityHider.toggleEntity(player, sheep);
}
}, 10 * TICKS_PER_SECOND);
}
return true;
}
}

finding the path of file in a Java program

In the following Java program (only showing a part of the whole program)
I have the follwing code:
private final File test_data_file;
private final int algorithm_selection;
private double average_pos_err, average_exe_time;
// The scan list to use for offline
private ArrayList<LogRecord> OfflineScanList;
public OfflineMode(RadioMap RM, File test_data_file, int algorithm_selection, Handler handler) {
this.RM = RM;
this.test_data_file = test_data_file;
this.handler = handler;
this.algorithm_selection = algorithm_selection;
this.OfflineScanList = new ArrayList<LogRecord>();
}
public void run() {
if (!test_data_file.isFile() || !test_data_file.exists() || !test_data_file.canRead()) {
errMsg = test_data_file + " does not exist or is not readable";
handler.sendEmptyMessage(-1);
return;
}
I want to track down, the path of the variable "test_data_file" of type File, but the code does not seems to show me any directions.
do you know where I can find it?
Thank you.

You can use File#getAbsolutePath or File#getCanonicalPath which will resolve the path fully (removing relative components)

using by java 7 you can get file type
String fileType = Files.probeContentType(test_data_file.getPath());
for more detail you can refers
http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#probeContentType%28java.nio.file.Path%29

How to pass a custom class between Activities

I found many simple solutions to this (such as Intent.putExtra(String, String) and Bundle.putString(String, String)), but this is not helpful for my situation.
I have a class called MyMP3 which contains non-primitive types. I need to pass the following for MyMP3...
private AudioFile audioFile;
private Tag tag;
private int index;
private boolean saved, startedWithLyrics;
private String id3lyrics;
AudioFile and Tag are both classes that I imported from a .jar file. How can I go about passing these to another Activity via Intents? I tried messing with implementing Parcelable for my "MyMP3" class, but I am not sure how to correctly use these methods when not passing primitive types.
Could you help me out and look at my code below and try to tell me how to correctly use Parcelable with a custom class like mine? How do I set the Parcel in the writeToParcel function and how do I correctly retrieve the class in another Activity?
Below is my code (the part that is important, at least). I've been trying different things for a couple of days now, but I cannot get it to work. Please help me out!
public class MyMP3 extends AudioFile implements Parcelable
{
private AudioFile audioFile;
private Tag tag;
private int index;
private boolean saved, startedWithLyrics;
private String id3lyrics;
public MyMP3(File f, int index)
{
this.audioFile = AudioFileIO.read(f);
this.tag = this.audioFile.getTag();
this.index = index;
this.saved = false;
this.id3lyrics = getLyrics();
}
#Override
public int describeContents()
{
return 0;
}
#Override
public void writeToParcel(Parcel out, int flats)
{
/* This method does not work, but I do not know how else to implement it */
Object objects[] = {this.audioFile, this.tag, this.index, this.saved, this.startedWithLyrics, this.id3lyrics};
out.writeArray(objects);
}
public static final Parcelable.Creator<MyMP3> CREATOR = new Parcelable.Creator<MyMP3>()
{
public MyMP3 createFromParcel(Parcel in)
{
/* Taken from the Android Developer website */
return new MyMP3(in);
}
public MyMP3[] newArray(int size)
{
/* Taken from the Android Developer website */
return new MyMP3[size];
}
};
private MyMP3(Parcel in)
{
/* This method probable needs changed as well */
Object objects[] = in.readArray(MyMP3.class.getClassLoader());
}
}

You can make your MyMP3 class Parcelable like that. Make sure you get the read/write order correct. The non-primitives must also be Parcelable, so you might not have control over that unfortunately. Alternatively, you could come up with your own serialization/deserialization. You could use a text format, like JSON or XML. Another alternative is to use subclass Application (make sure you declare it in your manifest) and use it is as a place to hang objects that span Activities. This keeps the object in memory for the lifecycle of your app, so be careful with doing this.