logger.debug("There are still "+numOfFutures+" backgroundTasks waiting to be executed or have their status checked..");
return;
}
// Here, the above may have given a result of < 0 >, but a new task may be asked for execution right next and still await for execution..
// The crawling-jobs can be safely finish, by avoiding to shut-down as long as at least one worker is still running (waiting for the Controller to verify that the assignments-batch is completed).
@ -161,8 +164,11 @@ public class ScheduledTasks {
// So the Controller will now shut down if either of takes-types have not finished.
// Check whether there are any active bulk-import procedures.
logger.debug("There are still "+numOfBulkImportDirsUnderProcessing+" bulkImportDirsUnderProcessing..");
return;
}
// Check whether the workers have not shutdown yet, which means that they either crawl assignments or/and they are waiting for the Controller to process the WorkerReport and then shutdown.
// There might be the case that the Controller has not sent shutDown requests to the Workers yet, or it has, BUT:
// 1) A worker requests for new assignments before the shutDown request is handled by its side.
// 1) A worker requests for new assignments, before it can handle the shutDown request given to it.
// 2) A new Worker joins the Service (unexpected, but anyway).
StringwarnMsg="The Service is about to shutdown, after all under-processing assignments and/or bulkImport requests are handled. No new requests are accepted!";
logger.warn(warnMsg);// It's likely not an actual error, but still it's not accepted.
// In case the Controller is running inside a docker container, and we want to send the "shutdownServiceRequest" from the terminal (with curl), without entering inside the container,
@ -45,7 +50,7 @@ public class ShutdownServiceImpl implements ShutdownService {
if(!workerInfo.getHasShutdown())// A worker may have shutdown on its own (by sending it a shutDown request manually), so it will have told the Controller when it shut down. In case of a Worker-crash, the Controller will not know about it.
@ -534,7 +537,7 @@ public class UrlsServiceImpl implements UrlsService {
logger.error("Could not find any info for worker with id: \""+workerId+"\".");
returnfalse;
}
Stringurl="http://"+workerInfo.getWorkerIP()+":1881/api/addReportResultToWorker/" +assignmentRequestCounter;// This workerIP will NOT be null.
Stringurl="http://"+workerInfo.getWorkerIP()+":" +workerPort+"/api/addReportResultToWorker/" +assignmentRequestCounter;// This workerIP will NOT be null.
if(logger.isTraceEnabled())
logger.trace("Going to \"postReportResultToWorker\": \""+workerId+"\", for assignments_"+assignmentRequestCounter+((errorMsg!=null)?"\nError: "+errorMsg:""));
logger.debug("The assignments_"+assignmentsBatchCounter+" have "+numAllFullTexts+" distinct non-already-uploaded fullTexts (total is: "+numFullTextsFound.get()+"). Going to request them from the Worker \""+workerId+"\", in "+numOfBatches+" batches ("+numOfFullTextsPerBatch+" files each).");
// Check if one full text is left out because of the division. Put it int the last batch.