Exemples Java BlockingQueue
En Java, nous pouvons utiliserBlockingQueue pour créer une file d'attente partagée par le producteur et le consommateur.
-
Producteur - Générez des données et placez-les dans la file d'attente.
-
Consommateur - Supprimez les données de la file d'attente.
Note
Lisez cet article pour comprendre ce qu'estproducer and consumer.
Les implémentations BlockingQueue sont thread-safe, peuvent être utilisées en toute sécurité avec plusieurs producteurs et plusieurs consommateurs.
1. BlockingQueue
Un simple exemple deBlockingQueue, un producteur génère des données et les met dans une file d'attente, en même temps, un consommateur prend les données de la même file d'attente.
1.1 Producer – A Runnable object to put 20 integers into a queue.
ExecutorExample1.java
package com.example.concurrency.queue.simple.raw;
import java.util.concurrent.BlockingQueue;
public class Producer implements Runnable {
private final BlockingQueue queue;
@Override
public void run() {
try {
process();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
private void process() throws InterruptedException {
// Put 20 ints into Queue
for (int i = 0; i < 20; i++) {
System.out.println("[Producer] Put : " + i);
queue.put(i);
System.out.println("[Producer] Queue remainingCapacity : " + queue.remainingCapacity());
Thread.sleep(100);
}
}
public Producer(BlockingQueue queue) {
this.queue = queue;
}
}
1.2 Consumer – A Runnable object to take items from a queue.
Consumer.java
package com.example.concurrency.queue.simple.raw;
import java.util.concurrent.BlockingQueue;
public class Consumer implements Runnable {
private final BlockingQueue queue;
@Override
public void run() {
try {
while (true) {
Integer take = queue.take();
process(take);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
private void process(Integer take) throws InterruptedException {
System.out.println("[Consumer] Take : " + take);
Thread.sleep(500);
}
public Consumer(BlockingQueue queue) {
this.queue = queue;
}
}
1.3 Run it. Démarrez 1 producteur et 1 consommateur et créez une file d'attente de taille 10.
Main.java
package com.example.concurrency.queue.simple;
import com.example.concurrency.queue.simple.raw.Consumer;
import com.example.concurrency.queue.simple.raw.Producer;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
public class Main {
public static void main(String[] args) {
BlockingQueue queue = new LinkedBlockingQueue<>(10);
new Thread(new Producer(queue)).start();
new Thread(new Consumer(queue)).start();
}
}
Sortie
Le producteur n'essaiera pas de mettre plus de données dans la file d'attente si elle est pleine.
[Producer] Put : 0 [Producer] Queue remainingCapacity : 9 [Consumer] Take : 0 [Producer] Put : 1 [Producer] Queue remainingCapacity : 9 [Producer] Put : 2 [Producer] Queue remainingCapacity : 8 [Producer] Put : 3 [Producer] Queue remainingCapacity : 7 [Producer] Put : 4 [Producer] Queue remainingCapacity : 6 [Producer] Put : 5 [Producer] Queue remainingCapacity : 5 [Consumer] Take : 1 [Producer] Put : 6 [Producer] Queue remainingCapacity : 5 [Producer] Put : 7 [Producer] Queue remainingCapacity : 4 [Producer] Put : 8 [Producer] Queue remainingCapacity : 3 [Producer] Put : 9 [Producer] Queue remainingCapacity : 2 [Producer] Put : 10 [Producer] Queue remainingCapacity : 1 [Consumer] Take : 2 [Producer] Put : 11 [Producer] Queue remainingCapacity : 1 [Producer] Put : 12 [Producer] Queue remainingCapacity : 0 [Producer] Put : 13 [Consumer] Take : 3 [Producer] Queue remainingCapacity : 0 [Producer] Put : 14 [Consumer] Take : 4 [Producer] Queue remainingCapacity : 0 [Producer] Put : 15 [Consumer] Take : 5 [Producer] Queue remainingCapacity : 0 [Producer] Put : 16 [Consumer] Take : 6 [Producer] Queue remainingCapacity : 0 [Producer] Put : 17 [Consumer] Take : 7 [Producer] Queue remainingCapacity : 0 [Producer] Put : 18 [Consumer] Take : 8 [Producer] Queue remainingCapacity : 0 [Producer] Put : 19 [Consumer] Take : 9 [Producer] Queue remainingCapacity : 0 [Consumer] Take : 10 [Consumer] Take : 11 [Consumer] Take : 12 [Consumer] Take : 13 [Consumer] Take : 14 [Consumer] Take : 15 [Consumer] Take : 16 [Consumer] Take : 17 [Consumer] Take : 18 [Consumer] Take : 19
Le programme ne s'arrêtera pas ou ne quittera pas, il continuera à y courir pour mettre et prendre des données desBlockingQueue
2. BlockingQueue + Poison Pill
La «pilule empoisonnée» est une solution générale pour arrêter ou interrompre les fils des producteurs et des consommateurs. L'idée est que le producteur mette une «pilule empoisonnée» dans la file d'attente et quitte, si le «consommateur» voit la «pilule empoisonnée» puis s'arrête et sort.
2.1 A producer with poison pill solution.
ProducerPoison.java
package com.example.concurrency.queue.simple.poison;
import java.util.concurrent.BlockingQueue;
public class ProducerPoison implements Runnable {
private final BlockingQueue queue;
private final Integer POISON;
@Override
public void run() {
try {
process();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
while (true) {
try {
queue.put(POISON);
break;
} catch (InterruptedException e) {
//...
}
}
}
}
private void process() throws InterruptedException {
// Put 20 elements into Queue
for (int i = 0; i < 20; i++) {
System.out.println("[Producer] Put : " + i);
queue.put(i);
System.out.println("[Producer] Queue remainingCapacity : " + queue.remainingCapacity());
Thread.sleep(100);
}
}
public ProducerPoison(BlockingQueue queue, Integer POISON) {
this.queue = queue;
this.POISON = POISON;
}
}
2.2 A consumer with poison pill solution.
ConsumerPoison.java
package com.example.concurrency.queue.simple.poison;
import java.util.concurrent.BlockingQueue;
public class ConsumerPoison implements Runnable {
private final BlockingQueue queue;
private final Integer POISON;
@Override
public void run() {
try {
while (true) {
Integer take = queue.take();
process(take);
// if this is a poison pill, break, exit
if (take == POISON) {
break;
}
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
private void process(Integer take) throws InterruptedException {
System.out.println("[Consumer] Take : " + take);
Thread.sleep(500);
}
public ConsumerPoison(BlockingQueue queue, Integer POISON) {
this.queue = queue;
this.POISON = POISON;
}
}
2.3 Start 2 producers and 2 consumers.
Main.java
package com.example.concurrency.queue.simple;
import com.example.concurrency.queue.simple.poison.ConsumerPoison;
import com.example.concurrency.queue.simple.poison.ProducerPoison;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
public class Main {
public static void main(String[] args) {
BlockingQueue queue = new LinkedBlockingQueue<>(10);
//new Thread(new Producer(queue)).start();
//new Thread(new Consumer(queue)).start();
Integer poison = -1;
new Thread(new ProducerPoison(queue, poison)).start();
new Thread(new ProducerPoison(queue, poison)).start();
new Thread(new ConsumerPoison(queue, poison)).start();
new Thread(new ConsumerPoison(queue, poison)).start();
}
}
Sortie
[Producer] Put : 0 [Producer] Put : 0 [//... [Consumer] Take : 18 [Consumer] Take : 18 [Consumer] Take : 19 [Consumer] Take : 19 [Consumer] Take : -1 [Consumer] Take : -1 Process finished with exit code 0
3. BlockingQueue + File Indexing
Un exemple deBlockingQueue pour créer un moteur d'indexation de fichiers simple. Un producteur analyse un répertoire et place le nom de fichier dans la file d'attente, en même temps, le consommateur prend le nom de fichier de la même file d'attente et l'indexe.
3.1 Producer.
FileCrawlerProducer.java
package com.example.concurrency.queue.crawler;
import java.io.File;
import java.io.FileFilter;
import java.util.concurrent.BlockingQueue;
// Producer
// Crawl file system and put the filename in BlockingQueue.
public class FileCrawlerProducer implements Runnable {
private final BlockingQueue fileQueue;
private final FileFilter fileFilter;
private final File file;
private final File POISON;
private final int N_POISON_PILL_PER_PRODUCER;
@Override
public void run() {
try {
crawl(file);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
while (true) {
try {
System.out.println(Thread.currentThread().getName()
+ " - FileCrawlerProducer is done, try poison all the consumers!");
// poison all threads
for (int i = 0; i < N_POISON_PILL_PER_PRODUCER; i++) {
System.out.println(Thread.currentThread().getName() + " - puts poison pill!");
fileQueue.put(POISON);
}
break;
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
public FileCrawlerProducer(BlockingQueue fileQueue,
FileFilter fileFilter, File file,
File POISON, int n_POISON_PILL_PER_PRODUCER) {
this.fileQueue = fileQueue;
this.fileFilter = fileFilter;
this.file = file;
this.POISON = POISON;
N_POISON_PILL_PER_PRODUCER = n_POISON_PILL_PER_PRODUCER;
}
private void crawl(File root) throws InterruptedException {
File[] entries = root.listFiles(fileFilter);
if (entries != null) {
for (File entry : entries) {
if (entry.isDirectory()) {
crawl(entry);
} else if (!isIndexed(entry)) {
System.out.println("[FileCrawlerProducer] - Found..."
+ entry.getAbsoluteFile());
fileQueue.put(entry);
}
}
}
}
private boolean isIndexed(File f) {
return false;
}
}
3.2 Consumer.
IndexerConsumer.java
package com.example.concurrency.queue.crawler;
import java.io.File;
import java.util.concurrent.BlockingQueue;
// Consumer
public class IndexerConsumer implements Runnable {
private final BlockingQueue fileQueue;
private final File POISON;
@Override
public void run() {
try {
while (true) {
File take = fileQueue.take();
if (take == POISON) {
System.out.println(Thread.currentThread().getName() + " die");
break;
}
indexFile(take);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
public void indexFile(File file) {
if (file.isFile()) {
System.out.println(Thread.currentThread().getName()
+ " [IndexerConsumer] - Indexing..." + file.getAbsoluteFile());
}
}
public IndexerConsumer(BlockingQueue fileQueue, File POISON) {
this.fileQueue = fileQueue;
this.POISON = POISON;
}
}
3.3 Start 1 producer and 2 consumers.
Main.java
package com.example.concurrency.queue.crawler;
import java.io.File;
import java.io.FileFilter;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
public class Main {
private static final File POISON = new File("This is a POISON PILL");
public static void main(String[] args) {
int N_PRODUCERS = 1;
int N_CONSUMERS = 2;//Runtime.getRuntime().availableProcessors();
int N_POISON_PILL_PER_PRODUCER = N_CONSUMERS / N_PRODUCERS;
int N_POISON_PILL_REMAIN = N_CONSUMERS % N_PRODUCERS;
System.out.println("N_PRODUCERS : " + N_PRODUCERS);
System.out.println("N_CONSUMERS : " + N_CONSUMERS);
System.out.println("N_POISON_PILL_PER_PRODUCER : " + N_POISON_PILL_PER_PRODUCER);
System.out.println("N_POISON_PILL_REMAIN : " + N_POISON_PILL_REMAIN);
//unbound queue, no limit
BlockingQueue queue = new LinkedBlockingQueue<>();
FileFilter filter = new FileFilter() {
public boolean accept(File file) {
return true;
}
};
File root = new File("C:\\users");
for (int i = 0; i < N_PRODUCERS - 1; i++) {
new Thread(new FileCrawlerProducer(queue, filter, root,
POISON, N_POISON_PILL_PER_PRODUCER)).start();
}
new Thread(new FileCrawlerProducer(queue, filter, root, POISON,
N_POISON_PILL_PER_PRODUCER + N_POISON_PILL_REMAIN)).start();
for (int i = 0; i < N_CONSUMERS; i++) {
new Thread(new IndexerConsumer(queue, POISON)).start();
}
}
}
Sortie
//... [FileCrawlerProducer] - Found...C:\users\Public\Videos\desktop.ini Thread-2 [IndexerConsumer] - Indexing...C:\users\Public\Videos\desktop.ini Thread-0 - FileCrawlerProducer is done, try poison all the consumers! Thread-0 - puts poison pill! Thread-0 - puts poison pill! Thread-1 die Thread-2 die Process finished with exit code 0