I am very new to the GC platform and am trying to create an API in Java with two methods: one which returns a list of all of the files in a specific bucket and another which retrieves a specified file from that bucket. The goal is to be able to iterate the file list in order to download every file from the bucket. Essentially, I am wanting to mirror the contents of the bucket on an Android device, so the API will be called from a generated client library in an Android app.
My getFileList() method returns a ListResult object. How do I extract a list of files from this?
#ApiMethod(name = "getFileList", path = "getFileList", httpMethod = ApiMethod.HttpMethod.GET)
public ListResult getFileList(#Named("bucketName") String bucketName) {
GcsService gcsService = GcsServiceFactory.createGcsService(RetryParams.getDefaultInstance());
ListResult result = null;
try {
result = gcsService.list(bucketName, ListOptions.DEFAULT);
return result;
} catch (IOException e) {
return null; // Handle this properly.
}
}
Also, I am struggling to determine what the return type of my getFile() API method should be. I can’t use a byte array since return types cannot be simple types as I understand it. This is where I am with it:
#ApiMethod(name = "getFile", path = "getFile", httpMethod = ApiMethod.HttpMethod.GET)
public byte[] getFile(#Named("bucketName") String bucketName, ListItem file) {
GcsService gcsService = GcsServiceFactory.createGcsService();
GcsFilename gcsFilename = new GcsFilename(bucketName, file.getName());
ByteBuffer byteBuffer;
try {
int fileSize = (int) gcsService.getMetadata(gcsFilename).getLength();
byteBuffer = ByteBuffer.allocate(fileSize);
GcsInputChannel gcsInputChannel = gcsService.openReadChannel(gcsFilename, 0);
gcsInputChannel.read(byteBuffer);
return byteBuffer.array();
} catch (IOException e) {
return null; // Handle this properly.
}
}
I am lost in the Google documentation for this stuff and am concerned that I am coming at it from completely the wrong direction since all I am trying to do is securely download a bunch of files!
I can't give you a complete solution because, this is code I wrote for my company, but I can show you some basics. I use the google-cloud-java API.
First you need to create an API key and download this in JSON format. More details can be found here.
I have - amongst other - these two fields in my class:
protected final Object storageInitLock = new Object();
protected Storage storage;
First you will need a method to initialize a com.google.cloud.storage.Storage object, something like (set your project-id and path to json api key):
protected final Storage getStorage() {
synchronized (storageInitLock) {
if (null == storage) {
try {
storage = StorageOptions.newBuilder()
.setProjectId(PROJECTID)
.setCredentials(ServiceAccountCredentials.fromStream(new FileInputStream(pathToJsonKey)))
.build()
.getService();
} catch (IOException e) {
throw new MyCustomException("Error reading auth file " + pathToJsonKey, e);
} catch (StorageException e) {
throw new MyCustomException("Error initializing storage", e);
}
}
return storage;
}
}
to get all entries you could use something like:
protected final Iterator<Blob> getAllEntries() {
try {
return getStorage().list(bucketName).iterateAll();
} catch (StorageException e) {
throw new MyCustomException("error retrieving entries", e);
}
}
list files in a directory:
public final Optional<Page<Blob>> listFilesInDirectory(#NotNull String directory) {
try {
return Optional.ofNullable(getStorage().list(getBucketName(), Storage.BlobListOption.currentDirectory(),
Storage.BlobListOption.prefix(directory)));
} catch (Exception e) {
return Optional.empty();
}
}
get info about a file:
public final Optional<Blob> getFileInfo(#NotNull String bucketFilename) {
try {
return Optional.ofNullable(getStorage().get(BlobId.of(getBucketName(), bucketFilename)));
} catch (Exception e) {
return Optional.empty();
}
}
adding a file:
public final void addFile(#NotNull String localFilename, #NotNull String bucketFilename,
#Nullable ContentType contentType) {
final BlobInfo.Builder builder = BlobInfo.newBuilder(BlobId.of(bucketName, bucketFilename));
if (null != contentType) {
builder.setContentType(contentType.getsValue());
}
final BlobInfo blobInfo = builder.build();
try (final RandomAccessFile raf = new RandomAccessFile(localFilename, "r");
final FileChannel channel = raf.getChannel();
final WriteChannel writer = getStorage().writer(blobInfo)) {
writer.write(channel.map(FileChannel.MapMode.READ_ONLY, 0, channel.size()));
} catch (Exception e) {
throw new MyCustomException(MessageFormat.format("Error storing {0} to {1}", localFilename,
bucketFilename), e);
}
}
I hope these code snippets and the referenced documentation will get you going, actulally it's not too hard.
Related
I have a Jhipster application that generate PDF with iText library, this PDF is saved in the computer in the route that I indicated.
I would like that when generating the pdf, a dialog box will appear to download the pdf. I am indifferent if the pdf is saved in the project folder or not saved in any place.
I have seen many posts giving possible answers on this page and on the internet, but many are already obsolete and others have not worked for me.
generatePDF
public void generatePDF(User u) {
String dest = "D:/PDF/result.pdf";
String src = "D:/PDF/template.pdf";
try {
PdfDocument pdf = new PdfDocument(new PdfReader(src), new PdfWriter(dest));
PdfAcroForm form = PdfAcroForm.getAcroForm(pdf, true);
Map<String, PdfFormField> fields = form.getFormFields();
fields.get("name").setValue(u.getFirstName());
fields.get("surname").setValue(u.getLastName());
fields.get("email").setValue(u.getEmail());
pdf.close();
} catch (IOException e) {
log.debug(e.getMessage());
}
}
UserResource
#GetMapping("/print-user/{id}")
#Timed
public ResponseEntity<User> printUserTemplate(#PathVariable Long id) {
User user = userRepository.findOne(id);
userService.generatePDF(user);
return ResponseUtil.wrapOrNotFound(Optional.ofNullable(user));
}
EDIT
entity.component.ts
downloadFile() {
this.entityService.downloadFile().subscribe();
}
entity.service.ts
downloadFile(): Observable<any> {
return this.http.get(SERVER_API_URL + 'api/downloadFile');
}
Use this to download the file:
#GetMapping("/downloadFile")
public ResponseEntity<Resource> downloadFile(HttpServletRequest request) {
// Load file as Resource
Resource resource = testService.loadFileAsResource();
// Try to determine file's content type
String contentType = null;
try {
contentType = request.getServletContext().getMimeType(resource.getFile().getAbsolutePath());
} catch (IOException ex) {
log.info("Could not determine file type.");
}
// Fallback to the default content type if type could not be determined
if (contentType == null) {
contentType = "application/octet-stream";
}
return ResponseEntity.ok().contentType(MediaType.parseMediaType(contentType)).header(
HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + resource.getFilename() + "\"").body(resource);
}
And this to generate the file:
public Resource loadFileAsResource() {
try {
Path path = Paths.get("D:\\PDF\\template.pdf");
Path filePath = path.normalize();
Resource resource = new UrlResource(filePath.toUri());
if (resource.exists()) {
return resource;
} else {
return null;
}
} catch (MalformedURLException ex) {
ex.printStackTrace();
return null;
}
}
References:
https://www.callicoder.com/spring-boot-file-upload-download-rest-api-example/
download a file from Spring boot rest service
I'm trying to append JSONObjects inside a JSONArray that is called Records .
The first time I save it it saves it this way that is ok
{
"Records": [
{
"travelTime": 2,
"totalDistance": 0,
"pace": 0,
"kCalBurned": 0,
"latlng": "[lat\/lng: (-32.1521234,-63.66412321)]"
}
]
}
But when I try to append again a new jsonobject inside Records, it creates a new JSONArray for it, and I just want to append a new object inside records
{
"Records": [
{
"travelTime": 2,
"totalDistance": 0,
"pace": 0,
"kCalBurned": 0,
"latlng": "[lat\/lng: (-31.6432292,-63.3667462)]"
}
]
}{
"Records": [
{
"travelTime": 1,
"totalDistance": 0,
"pace": 0,
"kCalBurned": 0,
"latlng": "[lat\/lng: (-31.9522431,-64.3461241)]"
}
]
}
This is the code I use to save the Records
private void writeJsonData(long travelTime,float totalDistance, float pace, float kCalBurned, LinkedList<LatLng> latlng){
String jsonStr = "";
JSONObject records = new JSONObject();
try {
records.put("travelTime", travelTime);
records.put("totalDistance", totalDistance);
records.put("pace", pace);
records.put("kCalBurned", kCalBurned);
records.put("latlng", latlng);
JSONArray jsonArray = new JSONArray();
jsonArray.put(records);
JSONObject recordsObj = new JSONObject();
recordsObj.put("Records", jsonArray);
jsonStr = recordsObj.toString();
} catch (JSONException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
String file_name = "records.json";
FileOutputStream fileOutputStream = null;
try {
fileOutputStream = new FileOutputStream(new File(mContext.getFilesDir(),file_name),true);
fileOutputStream.write(jsonStr.getBytes());
fileOutputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
You need a JSON parser so that you can locate the "Records" array inside the file and place the new data there. I used the "json simple" library (jar can be found here: https://code.google.com/archive/p/json-simple/downloads).
First you parse the file:
JSONParser parser = new JSONParser();
JSONObject records = null;
try {
records = (JSONObject) parser.parse(new FileReader("records.json"));
} catch (ParseException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
Then you locate the Records JSONArray. In there you want to append the new record:
JSONArray r = (JSONArray) records.get("Records");
Create the new record:
JSONObject NewObj = new JSONObject();
NewObj.put("travelTime", travelTime);
NewObj.put("totalDistance", totalDistance);
NewObj.put("pace", pace);
NewObj.put("kCalBurned", kCalBurned);
NewObj.put("latlng", latlng);
Add the new record to the "Records" JSONArray:
r.add(NewObj);
Write to file:
try (FileWriter file = new FileWriter("records.json")) {
file.write(records.toJSONString());
} catch (IOException ex) {
ex.printStackTrace();
}
Passing 2nd parameter to true in FileOutputStream constructor will
append jsonObject at the end of file.
To append it with JSON array inside Records object, you've to read the file first, append the new JSON object and write it back to file.
Use GSON library for conversion between java class & jSON. So you don't have to create JSON object manually each time by putting each key-pair.
Create a Java class to hold whole Records object
public class Record
{
#SerializedName("Records")
private List<Object> recordsList;
public Record()
{
this. recordsList = new ArrayList<>();
}
public List<Object> getRecordsList()
{
return recordsList;
}
}
Now create JAVA Model class to hold travel info
public class Travel {
private Integer travelTime;
private Integer totalDistance;
private Integer pace;
private Integer kCalBurned;
private LinkedList<LatLng> latlng;
public Integer getTravelTime() {
return travelTime;
}
public void setTravelTime(Integer travelTime) {
this.travelTime = travelTime;
}
public Integer getTotalDistance() {
return totalDistance;
}
public void setTotalDistance(Integer totalDistance) {
this.totalDistance = totalDistance;
}
public Integer getPace() {
return pace;
}
public void setPace(Integer pace) {
this.pace = pace;
}
public Integer getKCalBurned() {
return kCalBurned;
}
public void setKCalBurned(Integer kCalBurned) {
this.kCalBurned = kCalBurned;
}
public LinkedList<LatLng> getLatlng() {
return latlng;
}
public void setLatlng(LinkedList<LatLng> latlng) {
this.latlng = latlng;
}
}
Here is utility class with a function to append new JSON inside Records object. It will check if directory & file are created otherwise will create both.If file exist, it will read the file, append the new JSON object to list and write it back into the same file. You can change the directory & file name with yours.
Note: This class is written in Kotlin. Here is reference how to setup Android Studio for Kotlin
class Logger {
companion object {
private const val LOG_FILE_FOLDER = "Logs"
private const val LOG_FILE_NAME = "transaction"
private const val DATE_FORMAT = "yyyy-MM-dd"
private val logFileName: String
#SuppressLint("SimpleDateFormat")
get() {
var fileName = LOG_FILE_NAME
val dateFormat = SimpleDateFormat(DATE_FORMAT)
fileName += "_" + dateFormat.format(Date()) + ".json"
return fileName
}
fun logFile(json: Any) {
try {
val directoryPath = Environment.getExternalStorageDirectory().path + "/" + LOG_FILE_FOLDER
val loggingDirectoryPath = File(directoryPath)
var loggingFile = File("$directoryPath/$logFileName")
if (loggingDirectoryPath.mkdirs() || loggingDirectoryPath.isDirectory) {
var isFileReady = true
var isNewFile = false
if (!loggingFile.exists()) {
isFileReady = false
try {
loggingFile.createNewFile()
isNewFile = true
isFileReady = true
} catch (e: Exception) {
e.printStackTrace()
}
} else {
val lastFile = getLastFile(loggingFile.name, directoryPath)
loggingFile = File("$directoryPath/$lastFile")
val fileSize = getFileSize(loggingFile)
}
if (isFileReady) {
var jsonString: String? = null
if (!isNewFile) {
//Get already stored JsonObject
val stream = FileInputStream(loggingFile)
try {
val fileChannel = stream.channel
val mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size())
jsonString = Charset.defaultCharset().decode(mappedByteBuffer).toString()
} catch (e: Exception) {
e.printStackTrace()
} finally {
stream.close()
}
}
//Create record object
val record = if (!jsonString.isNullOrEmpty()) {
Gson().fromJson(jsonString, Record::class.java)
} else {
Record()
}
//Append the current json
record.recordList.add(json)
//create json to save
val jsonToSave = Gson().toJson(record)
val bufferedOutputStream: BufferedOutputStream
try {
bufferedOutputStream = BufferedOutputStream(FileOutputStream(loggingFile))
bufferedOutputStream.write(jsonToSave.toByteArray())
bufferedOutputStream.flush()
bufferedOutputStream.close()
} catch (e4: FileNotFoundException) {
e4.printStackTrace()
} catch (e: IOException) {
e.printStackTrace()
} finally {
System.gc()
}
}
}
} catch (ex: Exception) {
ex.printStackTrace()
}
}
}
}
At the end, you can log the file withlogFile method
Logger.Companion.logFile(travel);
Cheers :)
Hi I'm trying to make a PACS server using Java. dcm4che appears to be quite popular. But I'm unable to find any good examples about it.
As a starting point I inspected dcmqrscp and it successfully stores a DICOM image. But I cannot manage to handle a C-MOVE call. Here's my CMove handler. It finds requested the DICOM file adds a URL and other stuff, it doesn't throw any exception yet client doesn't receive any files.
private final class CMoveSCPImpl extends BasicCMoveSCP {
private final String[] qrLevels;
private final QueryRetrieveLevel rootLevel;
public CMoveSCPImpl(String sopClass, String... qrLevels) {
super(sopClass);
this.qrLevels = qrLevels;
this.rootLevel = QueryRetrieveLevel.valueOf(qrLevels[0]);
}
#Override
protected RetrieveTask calculateMatches(Association as, PresentationContext pc, final Attributes rq, Attributes keys) throws DicomServiceException {
QueryRetrieveLevel level = QueryRetrieveLevel.valueOf(keys, qrLevels);
try {
level.validateRetrieveKeys(keys, rootLevel, relational(as, rq));
} catch (Exception e) {
e.printStackTrace();
}
String moveDest = rq.getString(Tag.MoveDestination);
final Connection remote = new Connection("reciverAE",as.getSocket().getInetAddress().getHostAddress(), 11113);
if (remote == null)
throw new DicomServiceException(Status.MoveDestinationUnknown, "Move Destination: " + moveDest + " unknown");
List<T> matches = DcmQRSCP.this.calculateMatches(keys);
if (matches.isEmpty())
return null;
AAssociateRQ aarq;
Association storeas = null;
try {
aarq = makeAAssociateRQ(as.getLocalAET(), moveDest, matches);
storeas = openStoreAssociation(as, remote, aarq);
} catch (Exception e) {
e.printStackTrace();
}
BasicRetrieveTask<T> retrieveTask = null;
retrieveTask = new BasicRetrieveTask<T>(Dimse.C_MOVE_RQ, as, pc, rq, matches, storeas, new BasicCStoreSCU<T>());
retrieveTask.setSendPendingRSPInterval(getSendPendingCMoveInterval());
return retrieveTask;
}
private Association openStoreAssociation(Association as, Connection remote, AAssociateRQ aarq)
throws DicomServiceException {
try {
return as.getApplicationEntity().connect(as.getConnection(),
remote, aarq);
} catch (Exception e) {
throw new DicomServiceException(
Status.UnableToPerformSubOperations, e);
}
}
private AAssociateRQ makeAAssociateRQ(String callingAET,
String calledAET, List<T> matches) {
AAssociateRQ aarq = new AAssociateRQ();
aarq.setCalledAET(calledAET);
aarq.setCallingAET(callingAET);
for (InstanceLocator match : matches) {
if (aarq.addPresentationContextFor(match.cuid, match.tsuid)) {
if (!UID.ExplicitVRLittleEndian.equals(match.tsuid))
aarq.addPresentationContextFor(match.cuid,
UID.ExplicitVRLittleEndian);
if (!UID.ImplicitVRLittleEndian.equals(match.tsuid))
aarq.addPresentationContextFor(match.cuid,
UID.ImplicitVRLittleEndian);
}
}
return aarq;
}
private boolean relational(Association as, Attributes rq) {
String cuid = rq.getString(Tag.AffectedSOPClassUID);
ExtendedNegotiation extNeg = as.getAAssociateAC().getExtNegotiationFor(cuid);
return QueryOption.toOptions(extNeg).contains(
QueryOption.RELATIONAL);
}
}
I added the code below to send a DICOM file as a response:
String cuid = rq.getString(Tag.AffectedSOPClassUID);
String iuid = rq.getString(Tag.AffectedSOPInstanceUID);
String tsuid = pc.getTransferSyntax();
try {
DcmQRSCP.this.as=as;
File f = new File("D:\\dcmqrscpTestDCMDir\\1.2.840.113619.2.30.1.1762295590.1623.978668949.886\\1.2.840.113619.2.30.1.1762295590.1623.978668949.887\\1.2.840.113619.2.30.1.1762295590.1623.978668949.888");
FileInputStream in = new FileInputStream(f);
InputStreamDataWriter data = new InputStreamDataWriter(in);
// !1! as.cmove(cuid,1,keys,tsuid,"STORESCU");
as.cstore(cuid,iuid,1,data,tsuid,rspHandlerFactory.createDimseRSPHandler(f));
} catch (Exception e) {
e.printStackTrace();
}
Throws this exception
org.dcm4che3.net.NoRoleSelectionException: No Role Selection for SOP Class 1.2.840.10008.5.1.4.1.2.2.2 - Study Root Query/Retrieve Information Model - MOVE as SCU negotiated
You should add a role to the application instance like:
applicationEntity.addTransferCapability(
new TransferCapability(null, "*", TransferCapability.Role.SCP, "*"));
i am trying to record and reab back my list into file. It's working great until I restart my application. I am working with simulator (I don't have a real phone under Android)
Here is my function to record my class into a file :
public boolean writeRecordsToFile(String path, DummyContent object){
FileOutputStream fos;
ObjectOutputStream oos = null;
try {
fos = fileContext.openFileOutput(path, Context.MODE_PRIVATE);
oos = new ObjectOutputStream(fos);
oos.writeObject(object);
oos.close();
Log.d("fileManager", "Records write successfully");
return true;
} catch (Exception e) {
Log.e("fileManager", "Cant save records : " + e.getMessage());
return false;
}
finally {
if (oos != null)
try {
oos.close();
} catch (Exception e) {
Log.e("fileManager", "Error while closing stream "+e.getMessage());
}
}
}
Here is my reading Function :
public boolean readRecordsFromFile(String path){
FileInputStream fin;
ObjectInputStream ois=null;
try {
fin = fileContext.openFileInput(path);
ois = new ObjectInputStream(fin);
DummyContent records = (DummyContent) ois.readObject();
records.addItem("test", "test", "test");
ois.close();
Log.d("fileManager", "Records read successfully :\n" + records.toString());
Log.d("fileManager", "nbArticle found : " + String.valueOf(records.ITEMS.size()));
Log.d("fileManager", "article 0 title :\n" + records.ITEMS.get(0).content);
Log.d("fileManager", "article 10 title :\n" + records.ITEMS.get(10).content);
return true;
} catch (Exception e) {
Log.e("fileManager", "Cant read saved records : "+e.getMessage());
return false;
}
finally {
if (ois != null)
try {
ois.close();
} catch (Exception e) {
Log.e("fileManager", "Error in closing stream while reading records : "+e.getMessage());
}
}
}
and here is my class :
public class DummyContent implements Serializable {
/**
* An array of sample (dummy) items.
*/
public static List<DummyItem> ITEMS = new ArrayList<DummyItem>();
/**
* A map of sample (dummy) items, by ID.
*/
public static Map<String, DummyItem> ITEM_MAP = new HashMap<String, DummyItem>();
public void addItem(String first, String second, String third) {
DummyItem dummyItem = new DummyItem(first, second, third, android.R.drawable.ic_input_add);
ITEMS.add(dummyItem);
ITEM_MAP.put(dummyItem.id, dummyItem);
}
public void deleteAll() {
ITEMS = new ArrayList<DummyItem>();
ITEM_MAP = new HashMap<String, DummyItem>();
}
public void changeURL(Long index, String newURL) {
ITEMS.get(index.intValue()).url = newURL;
}
public void changeContent(Long index, String newContent) {
ITEMS.get(index.intValue()).contenu = newContent;
}
/**
* A dummy item representing a piece of content.
*/
public static class DummyItem {
public final String id;
public final String content;
public final String details;
public final int imageResource;
public String url;
public String contenu;
public DummyItem(String id, String content, String details, int imageResource) {
this.id = id;
this.content = content;
this.details = details;
this.imageResource = imageResource;
this.url = "";
this.contenu = "";
}
#Override
public String toString() {
return content;
}
}
}
Finally I read my file at the onCreate of my MainActivity (first activity at the lunching app) :
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
fileManager = new FileManager(this.getApplicationContext());
Log.d("Main", String.valueOf(fileManager.fileExist("Article.art")));
fileManager.readRecordsFromFile("Article.art"); /* Bug here : size of my array is empty but file's size is the same */
}
Here is my console return :
D/fileManager: size = 102
D/fileManager: Records read successfully :
D/fileManager: nbArticle found : 1 (because i add an item at the read
function) E/fileManager: Cant read saved records : Index: 10, Size: 1
I know it is working because when I write and read directly after the writting, I got all my items and I can read several times and I still got all items (this bug semms to be only present when I restart my application)
Maybe can I got help ?
Thanks !
The reason is very easy: You have declared some field as static, which are not covered by standard serialization: So, the contents of these fields were never written nor read from the file. That's why they "dissapeared" after a JVM restart.
Any field you want to be serialized/deserialized, you must declare it as instance member (not static).
See documentation on Serializable.
Good day, everyone! I have a little question about testing and generating a stub for dependence through reflection. So let's assume I have a class named UnderTest:
class UnderTest{
/*field*/
SomeLogic someLogic;
/*method, that i testing*/
List<MyObject> getCalculatedObjects(params) {/*logic,based on result getSomeStuff of someLogic*/ }
}
class SomeLogic {
List<String> getSomeStuff(String param) { /*Some complex and slow code, actually don't want test this code, and want to use some reflection invocation handler*/ }
}
For me it's important to not change legacy code, which doesn't design for testing. And i don't have any reason, except testing to make SomeLogic as an interface and so on.
I can't remember how to handle method invocation of someLogic using reflection. But google search isn't helping me.
Class MainAPI is... main api of my module. NetworkProvider long open stream operation, that's why i want to stub it, on my local files. But don't using directly reference on NetworkProvider. Again sorry for my English.
public class MainAPI {
private final XPath xPath;
private final ItemParser itemParser;
private final ListItemsParser listItemsParser;
private final DateParser dateParser;
private final HtmlCleanUp htmlCleanUp;
private final NetworkProvider networkProvider;
public MainAPI(XPath xPath, ItemParser itemParser, ListItemsParser listItemsParser, DateParser dateParser, HtmlCleanUp htmlCleanUp, NetworkProvider networkProvider) {
this.xPath = xPath;
this.itemParser = itemParser;
this.listItemsParser = listItemsParser;
this.dateParser = dateParser;
this.htmlCleanUp = htmlCleanUp;
this.networkProvider = networkProvider;
}
public MainAPI() throws XPathExpressionException, IOException {
dateParser = new DateParser();
xPath = XPathFactory.newInstance().newXPath();
networkProvider = new NetworkProvider();
listItemsParser = new ListItemsParser(xPath, dateParser, item -> true);
itemParser = new ItemParser(xPath, dateParser, networkProvider);
htmlCleanUp = new HtmlCleanUpByCleaner();
}
public List<Item> getItemsFromSessionParsing(SessionParsing sessionParsing) {
listItemsParser.setCondition(sessionParsing.getFilter());
List<Item> result = new ArrayList<>();
Document cleanDocument;
InputStream inputStream;
for (int currentPage = sessionParsing.getStartPage(); currentPage <= sessionParsing.getLastPage(); currentPage++) {
try {
inputStream = networkProvider.openStream(sessionParsing.getUrlAddressByPageNumber(currentPage));
} catch (MalformedURLException e) {
e.printStackTrace();
break;
}
cleanDocument = htmlCleanUp.getCleanDocument(inputStream);
List<Item> list = null;
try {
list = listItemsParser.getList(cleanDocument);
} catch (XPathExpressionException e) {
e.printStackTrace();
break;
}
for (Item item : list) {
inputStream = null;
try {
inputStream = networkProvider.openStream("http://www.avito.ru" + item.getUrl());
} catch (MalformedURLException e) {
e.printStackTrace();
break;
}
cleanDocument = htmlCleanUp.getCleanDocument(inputStream);
try {
item.setDescription(itemParser.getDescription(cleanDocument));
} catch (XPathExpressionException e) {
e.printStackTrace();
}
}
result.addAll(list);
}
return result;
}
}
public class NetworkProvider {
private final ListCycleWrapper<Proxy> proxyList;
public NetworkProvider(List<Proxy> proxyList) {
this.proxyList = new ListCycleWrapper<>(proxyList);
}
public NetworkProvider() throws XPathExpressionException, IOException {
this(new ProxySiteParser().getProxyList(new HtmlCleanUpByCleaner().getCleanDocument(new URL("http://www.google-proxy.net").openStream())));
}
public int getSizeOfProxy() {
return proxyList.size();
}
public InputStream openStream(String urlAddress) throws MalformedURLException {
URL url = new URL(urlAddress);
while (!proxyList.isEmpty()) {
URLConnection con = null;
try {
con = url.openConnection(proxyList.getNext());
con.setConnectTimeout(6000);
con.setReadTimeout(6000);
return con.getInputStream();
} catch (IOException e) {
proxyList.remove();
}
}
return null;
}
}
All the dependencies of your class to tests are injectable using its constructor, so there shouldn't be any problem to stub these dependencies and injecting the stubs. You don't even need reflection. For example, using Mockito:
NetworkProvider stubbedNetworkProvider = mock(NetworkProvider.class);
MainAPI mainApi = new MainAPI(..., stubbedNetworkProvider);
You can also write a stub by yourself if you want:
NetworkProvider stubbedNetworkProvider = new NetworkProvider(Collections.emptyList()) {
// TODO override the methods to stub
};
MainAPI mainApi = new MainAPI(..., stubbedNetworkProvider);