Java - Serialization - Grabbing number of objects - java

Java - Serialization - Grabbing number of objects in file
I'm trying to retrieve my objects from my serialized file and re-add them to my file. There seems to be an issue, no exception is being thrown but nothing is being printed in my console when running my method. Before continuing here is my code:
public boolean openCollection(){
try {
FileInputStream e = new FileInputStream("profiles.ser");
ObjectInputStream inputStream = new ObjectInputStream(e);
List<Profile> profiles = (List<Profile>) inputStream.readObject();
//De-obscure
for(Profile p : profiles){
String unObcName = deobscure(p.getName()); //Original name
String unObcSurname = deobscure(p.getSurname()); //Original surname
String unObcUsername = deobscure(p.getUsername()); //Original username
String unObcPassword = deobscure(p.getPassword()); //Original password
p.setName(unObcName);
p.setSurname(unObcSurname);
p.setUsername(unObcUsername);
p.setPassword(unObcPassword);
//Debugging
System.out.println("DE-OBSCURE - Profile name: " + p.getName() +"\n"+
"Profile surname: " + p.getSurname() +"\n"+
"Profile username: " + p.getUsername() +"\n"+
"Profile password: " + p.getPassword());
this.profiles.add(p);
}
} catch (FileNotFoundException var3) {
SwingUtilities.invokeLater(new Runnable() {
#Override
public void run() {
JOptionPane.showMessageDialog(null, "No profiles found, please create a profile!");
final CreateProfile createProfile = new CreateProfile();
createProfile.setVisible(true);
}
});
return false;
} catch (IOException var4) {
var4.printStackTrace();
JOptionPane.showMessageDialog(null, "IO Exception");
return false;
} catch (ClassNotFoundException var5) {
var5.printStackTrace();
JOptionPane.showMessageDialog(null, "Required class not found");
return false;
}
return true;
}
This is the serialization method
public void saveCollection(){
//Obscure the data
List<Profile> saveProfiles = new ArrayList<>();
for(Profile p : profiles){
String obcName = obscure(p.getName());
String obcSurname = obscure(p.getSurname());
String obcUsername = obscure(p.getUsername());
String obcPassword = obscure(p.getPassword());
p.setName(obcName);
p.setSurname(obcSurname);
p.setUsername(obcUsername);
p.setPassword(obcPassword);
//Debugging
System.out.println("DEBUG - Profile name: " + p.getName() + "\n" +
"Profile surname: " + p.getSurname() + "\n" +
"Profile username: " + p.getUsername() + "\n" +
"Profile password: " + p.getPassword());
saveProfiles.add(p);
}
//Save it
try {
FileOutputStream e = new FileOutputStream("profiles.ser");
ObjectOutputStream outputStream = new ObjectOutputStream(e);
outputStream.writeObject(saveProfiles);
outputStream.flush();
outputStream.close();
} catch (IOException var3) {
var3.printStackTrace();
JOptionPane.showMessageDialog(null, "Error. Cannot save database.");
}
}
When creating a profile the details are being obscured properly, here are the results:
DEBUG - Profile name: OBF:1u2a1toa1w8v1tok1u30
Profile surname: OBF:1u2a1toa1w8v1tok1u30
Profile username: OBF:1u2a1toa1w8v1tok1u30
Profile password: OBF:1u2a1toa1w8v1tok1u30
However when running openCollection() nothing is being printed into the console.
NOTE: The profile details were all 'admin' which is why all the data looks the same

After a while checking things out I figured out that the issue was simply with my deobscuring method. I have now edited it and replace it with the following:
/**
* #param str Obscured String
* #return Unobscured String
*/
private String deobscure(String s){
if (s.startsWith(__OBFUSCATE)) s = s.substring(4);
byte[] b = new byte[s.length() / 2];
int l = 0;
for (int i = 0; i < s.length(); i += 4)
{
if (s.charAt(i)=='U')
{
i++;
String x = s.substring(i, i + 4);
int i0 = Integer.parseInt(x, 36);
byte bx = (byte)(i0>>8);
b[l++] = bx;
}
else
{
String x = s.substring(i, i + 4);
int i0 = Integer.parseInt(x, 36);
int i1 = (i0 / 256);
int i2 = (i0 % 256);
byte bx = (byte) ((i1 + i2 - 254) / 2);
b[l++] = bx;
}
}
return new String(b, 0, l,StandardCharsets.UTF_8);
}
Original Profile name: "Admin"
Obscured Profile name: "OBF:1npu1toa1w8v1tok1nsc
After de-obscuring Profile name: "Admin"
Thus the method now works as intended.

Related

Exception is not handled

When I open the view window, I enter the value of facultyCode and if I enter an existing value, then everything is fine, and if I enter a non-existent value, then the window freezes and nothing happens
CLIENT
#FXML
void initialize() {
showButton.setOnAction(actionEvent -> {
String input = null;
try {
socket = new Socket("127.0.0.1",3024);
outputStream = new DataOutputStream(socket.getOutputStream());
inputStream = new DataInputStream(socket.getInputStream());
} catch (IOException e) {
e.printStackTrace();
}
String facultyCode = facultyCodeArea.getText();
String output = "SELECT * FROM Faculty WHERE facultyCode = " + facultyCode + ";";
try {
outputStream.writeUTF(output);
outputStream.flush();
input = inputStream.readUTF(); //if the input is incorrect, input is not assigned to
//anything
} catch (IOException e) {
e.printStackTrace();
}
String[] dataServer = input.split(" ");
String name = dataServer[0];
nameArea.setText(name);
String code = dataServer[1];
facultyCodeArea.setText(code);
String number = dataServer[2];
numberSubjectsArea.setText(number);
String main = dataServer[3];
mainSubjectArea.setText(main);
String dean = dataServer[4];
deanArea.setText(dean);
String language = dataServer[5];
languageStudyArea.setText(language);
});
}
SERVER
else if (isSelectQuery(input)) {
statement = connection.createStatement();
ResultSet resultSet = statement.executeQuery(input);
while (resultSet.next()) {
String name = resultSet.getString("name");
int codeFaculty = resultSet.getInt("facultyCode");
int numberSubject = resultSet.getInt("numberSubjects");
String mainSubject = resultSet.getString("mainSubject");
String dean = resultSet.getString("dean");
String languageStudy = resultSet.getString("languageStudy");
String output = name + " " +
codeFaculty + " " +
numberSubject + " " +
mainSubject + " " +
dean + " " +
languageStudy;
outputStream.writeUTF(output);
outputStream.flush();
}
}
I've tried closing the window if an exception occurs, I've also tried closing the window if input = null, but didn't help
SERVER
String output = "error";
else if (isSelectQuery(input)) {
statement = connection.createStatement();
ResultSet resultSet = statement.executeQuery(input);
while (resultSet.next()) {
String name = resultSet.getString("name");
int codeFaculty = resultSet.getInt("facultyCode");
int numberSubject = resultSet.getInt("numberSubjects");
String mainSubject = resultSet.getString("mainSubject");
String dean = resultSet.getString("dean");
String languageStudy = resultSet.getString("languageStudy");
output = name + " " +
codeFaculty + " " +
numberSubject + " " +
mainSubject + " " +
dean + " " +
languageStudy;
}
outputStream.writeUTF(output);
outputStream.flush();
}
CLIENT
#FXML
void initialize() {
showButton.setOnAction(actionEvent -> {
String input = null;
try {
socket = new Socket("127.0.0.1",3024);
outputStream = new DataOutputStream(socket.getOutputStream());
inputStream = new DataInputStream(socket.getInputStream());
} catch (IOException e) {
e.printStackTrace();
}
String facultyCode = facultyCodeArea.getText();
String output = "SELECT * FROM Faculty WHERE facultyCode = " + facultyCode + ";";
try {
outputStream.writeUTF(output);
outputStream.flush();
input = inputStream.readUTF();
} catch (IOException e) {
e.printStackTrace();
}
String[] dataServer = input.split(" ");
if(!dataServer[0].equals("error")) {
String name = dataServer[0];
nameArea.setText(name);
String code = dataServer[1];
facultyCodeArea.setText(code);
String number = dataServer[2];
numberSubjectsArea.setText(number);
String main = dataServer[3];
mainSubjectArea.setText(main);
String dean = dataServer[4];
deanArea.setText(dean);
String language = dataServer[5];
languageStudyArea.setText(language);
}
else {
errorArea.setText("Not exist");
}
});
}
So I first assign output = error and if the request does not return anything, then I send it to the client, if on request something came, I assign values ​​and send it to the client. Next, I process the data on the client

HasMap Function returning null value

I working on android csv file reading library. I create a function in library and calling it in other project. The library function reads the csv file ( which is in the assets folder of my project which is extended from library class ) ,store the name and value in strings GotDbname and GotDbValue respectively . But it is not returning anything. Any type help will be appreciated.
public class ReadFile extends Activity {
String GetDbName = "DBName";
String GetDbVersion = "DbVersion";
String GotDbName;
String GotDbVersion;
HashMap<String, String> DatabaseMap = new HashMap<String, String>();
#Override
protected void onCreate(Bundle savedInstanceState) {
// TODO Auto-generated method stub
super.onCreate(savedInstanceState);
}
public void hello() {
Log.d("Library Is working ", "Hello ");
}
public HashMap<String, String> read(Context con, String name, String version) {
Log.d("Read File", "Getting Assest Manager");
AssetManager assetManager = con.getAssets();
Log.d("Read File", "Assest Manager Object Created");
if (assetManager == null) {
Log.d("Reading file", "Nothing in assets");
}
else {
InputStream inputStream = null;
Log.d("Read File", "Getting input Stream");
try {
Log.d("Read File", "Getting File");
inputStream = assetManager.open("Db.csv");
Log.d("Read File", "File read");
String text = loadTextFile(inputStream);
String[] text1 = text.split(",");
Log.d("Read File", "LoadTextFile" + text + " " + text1);
String id[] = new String[text1.length];
String value[] = new String[text1.length];
Log.d("Read File", "Getting Strings" + "DbName " + id
+ "DbVersion " + value);
int c = 0;
for (int i = 0; i < text1.length; i += 2)
{
id[c] = text1[i].replace('"', ' ');
Log.d("Read File", "Getting Db name" + " " + id[c]);
c++;
}
int d = 0;
for (int i = 1; i < text1.length; i += 2)
{
value[d] = text1[i].replace('"', ' ');
Log.d("Read File", "Getting getting Version" + " "
+ value[d]);
d++;
}
int indexofid = -1;
for (int i = 0; i < id.length; i++) {
Log.d("Reading CSV", "In for Loop");
if (GetDbName.equals(id[i].trim())) {
indexofid = i;
GotDbName = value[indexofid].toString();
Log.d("Read File final", "Getting Db Name" + " "
+ GotDbName);
break;
}
}
for (int i = 0; i < id.length; i++) {
if (GetDbVersion.equals(id[i].trim())) {
indexofid = i;
GotDbVersion = value[indexofid].toString();
Log.d("Read File final", "Getting Db Version" + " "
+ GotDbVersion);
break;
}
}
} catch (IOException e) {
Log.d("Read File", "No file found");
}
}
DatabaseMap.put(name, GotDbName);
DatabaseMap.put(version, GotDbVersion);
Log.d("Read File", " Database Name =" + GotDbName
+ "Database Version =" + GotDbVersion);
return DatabaseMap;
}
public String loadTextFile(InputStream inputStream) throws IOException {
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
byte[] bytes = new byte[4096];
int len = 0;
while ((len = inputStream.read(bytes)) > 0)
byteStream.write(bytes, 0, len);
return new String(byteStream.toByteArray(), "UTF8");
}
}
And here is the class of other project which is using that library
public class MainClass extends ReadFile {
String DbName;
String DbVersion;
String a, b;
TextView txt_dbName, txt_version;
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
txt_dbName = (TextView) findViewById(R.id.txt_dbName);
txt_version = (TextView) findViewById(R.id.txt_version);
HashMap<String, String> map = new HashMap<>();
map = read(this, DbName, DbVersion);
String abc = map.get("name");
Log.d("Main Class", "Map =" + abc);
Log.d("MainClass", "DbName = " + DbName + ": DbVersion = "
+ DbVersion);
txt_dbName.setText(DbName);
txt_version.setText(DbVersion);
}
}
String DbName;
String DbVersion;
...
map = read(this, DbName, DbVersion);
//The DbName and DbVersion is null or empty?
public HashMap<String, String> read(Context con, String name, String version) {
...
DatabaseMap.put(name, GotDbName); // If name is null, how to get?
DatabaseMap.put("name", GotDbName); // Maybe
DatabaseMap.put(version, GotDbVersion);
Log.d("Read File", " Database Name =" + GotDbName
+ "Database Version =" + GotDbVersion);
return DatabaseMap;
}

Splitting String based on this six characters 0102**

How can I split a flat string based on 0102**? string tokenizer is working for only **. Is there any way to split based on 0102**? Please suggest
Here is my complete method
private String handleCibil(InterfaceRequestVO ifmReqDto, String szExtIntType) throws MalformedURLException, org.apache.axis.AxisFault, RemoteException {
/* Declaration and initiliazation */
ConfVO confvo = ifmReqDto.getExtConfVo();
String szResponse = null;
String cibilResponse = null;
String errorResponse = null;
String endpointURL = null;
long timeOut = confvo.getBurMgr().getBurInfo(szExtIntType).getTimeOut();
endpointURL = formWebServiceURL(confvo, szExtIntType);
URL url = new URL(endpointURL);
log.debug("Input xml for cibil "+ifmReqDto.getIfmReqXML());
BasicHttpStub stub= new BasicHttpStub(url,new org.apache.axis.client.Service());
szResponse = stub.executeXMLString(ifmReqDto.getIfmReqXML());
//szResponse=szResponse.replaceAll("&", "&");
log.debug("szResponse "+szResponse);
/* Validate if the obtained response is as expected by IFM */
try {
extDao = new ExtInterfaceXMLTransDAO(ifmReqDto.getSemCallNo(), ifmReqDto.getIdService());
extDao.updateRqstRespXML10g(ifmReqDto.getInterfaceReqNum(), szResponse, GGIConstants.IFM_RESPONSE);
//log.debug("CIBIL_RESPONSE_XPATH " + GGIConstants.CIBIL_RESPONSE_XPATH);
Document xmlDocument = DocumentHelper.parseText(szResponse);
String xPath = GGIConstants.RESPONSE_XPATH;
List<Node> nodes = xmlDocument.selectNodes(xPath);
for (Node node : nodes) {
String keyValue = node.valueOf(GGIConstants.RESPONSE_XPATH_KEY);
// log.debug("keyValue : " + keyValue);
if (keyValue.equalsIgnoreCase(GGIConstants.RESPONSE_XPATH_KEY_VALUE)) {
// log.debug("node value : " + node.getText());
cibilResponse = node.getText();
}
}
log.debug("cibilResponse " + cibilResponse);
String errorResponseXPATH = GGIConstants.CIBIL_ERROR_RESPONSE_XPATH;
List<Node> errorResponseNode = xmlDocument.selectNodes(errorResponseXPATH);
for (Node node : errorResponseNode) {
errorResponse = node.getText();
}
log.debug("errorResponse " + errorResponse);
if(cibilResponse!=null && cibilResponse.length()>0)
{
StringTokenizer cibilResponseResults = new StringTokenizer(cibilResponse,"**");
String tempResponse="";
ArrayList probableMatchList = new ArrayList();
while (cibilResponseResults.hasMoreElements()) {
tempResponse = (String) cibilResponseResults.nextElement();
if(tempResponse.length()>=80)
{
String memberRefNo = tempResponse.substring(69, 80).replaceAll(" ", "");
log.debug("memberRefNo " + memberRefNo);
if (memberRefNo.length() > 0) {
if (Integer.parseInt(memberRefNo) > 0) {
cibilResponse = tempResponse;
cibilResponse = cibilResponse+"**";
}
else
{
probableMatchList.add(tempResponse+"**");
}
}
else
{
probableMatchList.add(tempResponse+"**");
}
}
else
{
cibilResponse = tempResponse+"**";
}
}
log.debug("After finding the Member reference number cibilResponse " + cibilResponse);
log.debug("After finding the Probable reference list " + probableMatchList);
// TKN 008
cibilResponse=StringEscapeUtils.unescapeXml(cibilResponse).replaceAll("[^\\x20-\\x7e]","");
ifmReqDto.setIfmTransformedResult(cibilResponse);
ifmReqDto.setProbableMatchList(probableMatchList);
}
if (errorResponse!=null && errorResponse.length()>0) {
throw new GenericInterfaceException(errorResponse
+ " for the seq_request " + ifmReqDto.getSeqRequest() + " Seq_Interface_req is >> "
+ ifmReqDto.getInterfaceReqNum(),
GGIConstants.SEND_REQUEST_CONSTANT + Strings.padStart(String.valueOf(ifmReqDto.getIdService()), 2, GGIConstants.DEFAULT_NUMBER_STRING)
+ GGIConstants.CIBIL_ERROR_CODE);
}
else if (cibilResponse==null || StringUtils.isEmpty(cibilResponse) ) {
throw new GenericInterfaceException("Cibil TUEF response is empty >> cibil Service "
+ "for the seq_request " + ifmReqDto.getSeqRequest() + "Seq_Interface_req is >> "
+ ifmReqDto.getInterfaceReqNum(),
GGIConstants.SEND_REQUEST_CONSTANT + Strings.padStart(String.valueOf(ifmReqDto.getIdService()), 2, GGIConstants.DEFAULT_NUMBER_STRING)
+ GGIConstants.INTERFACE_ERROR_RESPONSE);
}
/* Setting Instinct response to ifmReqDto object */
} catch (SQLException e) {
log.error("SQLException while connecting to DataBase. Exception message is ", e);
throw new GenericInterfaceException("SQLException >> Instinct Service "
+ "for the seq_request " + ifmReqDto.getSeqRequest() + "Seq_Interface_req is >> "
+ ifmReqDto.getInterfaceReqNum(),
GGIConstants.SEND_REQUEST_CONSTANT + Strings.padStart(String.valueOf(ifmReqDto.getIdService()), 2, GGIConstants.DEFAULT_NUMBER_STRING)
+ GGIConstants.DB_OPERATION_ERROR);
} catch (GenericInterfaceException exp) {
log.error("Exception occured while valid:", exp);
throw exp;
} catch (Exception exp) {
log.error("Exception occured while valid:", exp);
throw new GenericInterfaceException("GeneralException >> Instinct Service "
+ "for the seq_request " + ifmReqDto.getSeqRequest() + "Seq_Interface_req is >> "
+ ifmReqDto.getInterfaceReqNum(),
GGIConstants.SEND_REQUEST_CONSTANT + Strings.padStart(String.valueOf(ifmReqDto.getIdService()), 2, GGIConstants.DEFAULT_NUMBER_STRING)
+ GGIConstants.UNKNOWN_ERROR);
}
return szResponse;
}
I recommend checking out the Java documentation, it provides a really good reference to start with. The .split method uses a regex to split up a string based on a delimiter.
String[] tokens = myString.split("0102\\*\\*");
For now I suspect that you forgot to escape * in split regex.
Try maybe
String[] resutl = yourString.split("0102\\*\\*");
In case you want * to represent any character then use . instead of *
String[] resutl = yourString.split("0102..");
In case you want * to represent any digit use \\d instead
String[] resutl = yourString.split("0102\\d\\d");
String string = "blabla0102**dada";
String[] parts = string.split("0102\\*\\*");
String part1 = parts[0]; // blabla
String part2 = parts[1]; // dada
Here we have a String: "blabla0102**dada", we call it string. Every String object has a method split(), using this we can split a string on anything we desire.
Do you mean literally split by "0102**"? Couldn't you use regex for that?
String[] tokens = "My text 0102** hello!".split("0102\\*\\*");
System.out.println(tokens[0]);
System.out.println(tokens[1]);

create a zip file. and manage the output stream

I created a button that allows to create a zip file., the function that zips the file works correctly, but when I call her via the button (in the JS file) it crashes and it gives a blank page (I think I do not manage the output stream)
would please an idea
here is my code:
Button
isc.ToolStripButton.create({
ID: "BooksApp_GetXmlImage_Button"
,autoDraw:false
,icon: getUIIcon("icon_xml_16")
,prompt: getUIMsg("book_report_get_xml",4)
,showHover:true
,hoverStyle:"book_hover_style"
,click : function () {
BooksApp_Action_loadFile("objx");
// isc.say("test");
}
});
function to call the zipfile() method:
function BooksApp_Action_loadFile(p_UsedFormat) {
var tmpBookID = BooksApp_Application.FP_BookID;
var tmpIDs = BooksApp_Application.FP_fct_getSelectedPOVIDs();
var tmpUsr_ID = FPIUser.FP_fct_getID();
var tmpFormat = p_UsedFormat;
var showInWindow=false;
books_objects.exportData(
{
r_book_idnum : tmpBookID
,sBook_ID : tmpBookID
,sPOV_IDs : tmpIDs
,sUser_ID : tmpUsr_ID
,sFormat : tmpFormat
}
,{ operationId: "customExport"
,exportDisplay: (showInWindow ? "window" : "download") }
,function (dsResponse, data, dsRequest) {
//Never called
BooksApp_Action_Log("BooksApp_Action_loadFile:"+data);
}
);
}
customExport() function
public static String customExport(RPCManager rpc,
HttpServletResponse response) throws Exception {
String sReturn = _Return_OK;
try {
// setting doCustomResponse() notifies the RPCManager that we'll
// bypass RPCManager.send
// and instead write directly to the servletResponse output stream
rpc.doCustomResponse();
RequestContext.setNoCacheHeaders(response);
writeServerDebug("customExport : start");
DSRequest req = rpc.getDSRequest();
List<?> results = req.execute().getDataList();
String sReqData = (String) req.getParameter("exportDisplay");
String sReqData_sBook_ID = "" + req.getCriteriaValue("sBook_ID");
String sReqData_sPOV_IDs = "" + req.getCriteriaValue("sPOV_IDs");
String sReqData_sUser_ID = "" + req.getCriteriaValue("sUser_ID");
String sReqData_sFormat = "" + req.getCriteriaValue("sFormat");
StringBuilder content = new StringBuilder("get (sReqData:"
+ sReqData + ",sBook_ID:" + sReqData_sBook_ID
+ ",sPOV_IDs:" + sReqData_sPOV_IDs + ",sUser_ID:"
+ sReqData_sUser_ID + ",sFormat:" + sReqData_sFormat + ")"
+ results.size() + " line(s):");
for (Iterator<?> i = results.iterator(); i.hasNext();) {
Map<?, ?> record = (Map<?, ?>) i.next();
content.append("\n" + Books.Column_IDNum + ":"
+ record.get(Books.Column_IDNum));
content.append("\n" + Books.Column_Name + ":"
+ record.get(Books.Column_Name));
}
writeServerDebug("The content is \n" + content.toString());
// Create the new Office Engine
OfficeEngine myOfficeEngine = new OfficeEngine();
boolean bIsConnected = myOfficeEngine._RepositoryBridge
.connectSourceDataBase(false);
if (bIsConnected) {
//Connected to the repository, so get the files
if (sReqData_sFormat.equalsIgnoreCase("pdf") || sReqData_sFormat.equalsIgnoreCase("pptx")) {
//The book end user format
String sReturnPptx = myOfficeEngine.performGeneratePptx(
req.getHttpServletRequest(), response,
sReqData_sBook_ID, sReqData_sPOV_IDs,
sReqData_sUser_ID, sReqData_sFormat);
writeServerDebug("customExport call performGeneratePptx, return is "
+ sReturnPptx);
}
else {
AppZip appZip = new AppZip();
appZip.ZipFile(" ", " ");
String r = "sReturn_OK";;
return r;
}
//Free the connection to repository
myOfficeEngine._RepositoryBridge.freeConnectionSource();
} else {
response.setContentType("text/plain");
response.addHeader("content-disposition",
"attachment; filename=book.txt");
ServletOutputStream os = response.getOutputStream();
os.print(content.toString());
os.flush();
}
} catch (Exception e) {
writeServerDebug("ERROR:" + e.getLocalizedMessage());
sReturn = Repository._Return_KO;
}
return sReturn;
}

Download a Large Number of Files Using the Java SDK for Amazon S3 Bucket

I have a large number of files that need to be downloaded from an S3 bucket. My problem is similar to this article except I am trying to run it in Java.
public static void main(String args[]) {
AWSCredentials myCredentials = new BasicAWSCredentials("key","secret");
TransferManager tx = new TransferManager(myCredentials);
File file = <thefile>
try{
MultipleFileDownload myDownload = tx.downloadDirectory("<bucket>", null, file);
System.out.println("Transfer: " + myDownload.getDescription());
System.out.println(" - State: " + myDownload.getState());
System.out.println(" - Progress: " + myDownload.getProgress().getBytesTransfered());
while (myDownload.isDone() == false) {
System.out.println("Transfer: " + myDownload.getDescription());
System.out.println(" - State: " + myDownload.getState());
System.out.println(" - Progress: " + myDownload.getProgress().getBytesTransfered());
try {
// Do work while we wait for our upload to complete...
Thread.sleep(500);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
}
} catch(Exception e){
e.printStackTrace();
}
}
This was adapted from the TransferManager class example for multiple upload. There are well over a 100,000 objects in this bucket. Any help would be great.
Please use the list() method to get a list of your files, then use the get() method to get each file.
class S3 extends AmazonS3Client {
final String bucket;
S3(String u, String p, String Bucket) {
super(new BasicAWSCredentials(u, p));
bucket = Bucket;
}
String get(String k) {
try {
final S3Object f = getObject(bucket, k);
final BufferedInputStream i = new BufferedInputStream(f.getObjectContent());
final StringBuilder s = new StringBuilder();
final byte[] b = new byte[1024];
for (int n = i.read(b); n != -1; n = i.read(b)) {
s.append(new String(b, 0, n));
}
return s.toString();
} catch (Exception e) {
log("Cannot get " + bucket + "/" + k + " from S3 because " + e);
}
return null;
}
String[] list(String d) {
try {
final ObjectListing l = listObjects(bucket, d);
final List<S3ObjectSummary> L = l.getObjectSummaries();
final int n = L.size();
final String[] s = new String[n];
for (int i = 0; i < n; ++i) {
final S3ObjectSummary k = L.get(i);
s[i] = k.getKey();
}
return s;
} catch (Exception e) {
log("Cannot list " + bucket + "/" + d + " on S3 because " + e);
}
return new String[]{};
}
}
TransferManager internally uses countdownlatch which makes me believe is does concurrent download (which seems the right way to do it). It makes sense to use it than get one file after other sequentially?

Categories