Taogen's Blog

Introduction

Computer users want to do more than one thing at a time. A single application does more than one things at a time called concurrent software.

The Java platform is designed to support concurrent programming. Since Java SE 5, the Java platform has also included high-level concurrency APIs.

Processes and Threads

In concurrent programming, there are two basic units of execution: process and threads. In the Java programming language, concurrent programming is mostly concerned with threads.

A computer system has many active processes and threads. If in computer systems that only have a single execution core, there is one thread actually executing at any given moment. Processing time for a single core is shared among processes and threads through an OS feature called time slicing.

It’s becoming more and more common for computer systems to have multiple processors or processors with multiple execution cores.

Processes

A process has a self-contained execution environment. A process generally has a complete, private set of basic run-time resources, for example, memory space.

Processes are often seen as synonymous with programs or applications. Most implementations of Java Virtual machines run as a single process. A Java application can create additional processes using ProcessBuilder object.

Threads

Threads are sometimes called lightweight processes. Both processes and threads provide an execution environment, but creating a new thread requires fewer resources than creating a new process.

Threads exist within a process, and every process has at least one thread. Threads share the process’s resources, including memory and open files. This makes for efficient, but potentially problematic, communication.

Difference between processes and threads

Threads exist within a process, and every process has at least one thread. Both processes and threads are units of execution, and they have an execution environment.

A process is a minimal resource assignment unit, but a thread is a minimal execution unit.

A process has a self-contained execution environment. No resources are shared between processes. But threads have a private resource from a process, and multiple threads in the same process also can share the resource of their process.

The Advantages and Disadvantages of Threads

Advantages

• Improve the performance of applications.
• Asynchronous workflows.
• Exploiting multiple processors.
• More responsive user interfaces.

Disadvantages

• Safety.
• Liveness.
• Performance. Thread introduces additional performance costs, for example, context switches, and synchronization costs.

Properly using multi-thread is more beneficial than disadvantages.

Thread Objects

Each thread is associated with an instance of the class Thread. There are two basic strategies for using Thread objects to create a concurrent application.

• To directly control thread creation and management, simply instantiate Thread each time.
• To abstract thread management from the rest of your application, pass the application’s task to an executor.

Creating an instance of Thread must provide the code that will run in that thread. There are two ways to create a Thread instance:

• Provide a runnable object.
• Subclass Thread.

Using a class of implemented Runnable interface to create an instance of Thread is more general because of the Runnable object can subclass a class other than Thread.

Methods of Thread class:

• Constructors
  • Thread(), Thread(String name)
  • Thread(Runnable target), Thread(Runnable target, String name)
  • Thread(ThreadGroup group, Runnable target, String name), Thread(ThreadGroup group, Runnable target, String name, long stackSize)
  • Thread(ThreadGroup group, String name)
• Static Methods
  • static in activeCount()
  • static Thread currentThread()
  • static void dumpStack(), static Map<Thread, StackTraceElement[]> getAllStackTraces()
  • static int enumerate(Thread[] tarray)
  • static Thread.UncaughtExceptionHandler getDefaultUncaughtExceptionHandler()
  • static boolean holdsLock(Object obj)
  • static boolean interrupted()
  • static void yield()
• Get Thread Information
  • long getId()
  • String getName()
  • int getPriority()
  • ClassLoader getConctextClassLoader()
  • StackTraceElement[] getStackTrace()
  • Thread.State getState()
  • ThreadGroup getThreadGroup()
  • Thread.UncaughtExceptionHandler getUncaughtExceptionHandler()
• Check
  • void checkAccess()
  • boolean isAlive()
  • boolean isDaemon()
  • boolean isInterrupted()
• Operating thread
  • void interrupt()
  • void join(), void join(long millis), join(long millis, int nanos)
  • void run()
  • void setName(String name), void setPriority(int newPriority), void setContextClassLoader(ClassLoader c), void setDaemon(boolean on)
  • void sleep(long millis), sleep(long millis, int nanos)
  • void start()

Why are Thread.stop, Thread.suspend and Thread.resume Deprecated?

Interrupts

An interrupt is an indication to a thread that it should stop what it is doing and do something else. Thread.interrupt() sets the interrupted status/flag of the target thread. The code running in that target thread MAY poll the interrupted status and handle it appropriately. Some methods that block such as Object.wait() may consume the interrupted status immediately and throw an appropriate exception (usually InterruptedException). Thread interruption is a gentle way to stop a thread. It is used to give threads a chance to exit cleanly, as opposed by deprecated Thread.stop() that force to stop the thread.

Interruption in Java is not preemptive. Threads have to cooperate in order to process the interrupt properly. If the target thread does not poll the interrupted status the interrupt is effectively ignored. Polling occurs via the Thread.interrupted() method which returns the current thread’s interrupted status and clears that interrupt flag. Usually, the thread might then do something such as throw InterruptedException. If a thread goes a long time without invoking a method that throws InterruptedException Then it must periodically invoke Thread.interrupted(), which returns if an interrupt has been received. For example:

while (...){
    doSometing();
    if (Thread.interrupted()){
        // We've been interrupted
        System.out.println("Thread interrupted!");
        return; // or break loop
    }
}

if (Thread.interrupted()){
    throw new InterrupteException();
}

Some API methods have built-in interrupt handling

• Object.wait()
• Thread.sleep(), Thread.join()
• Most java.util.concurrent structures.
• Java NIO (it does not use InterruptedException, instead using ClosedByInterruptException).

Joins

The join method allows one thread to wait for the completion of another. Example of current thread wait for thread t to be complete:

t.join()

join is similar to sleep doesn’t release the locks it holds, it just suspends the current thread.

Difference Between Wait and Sleep

• wait is for concurrent programming but sleep not. Sleeping does not release the locks it holds while wait releases the lock on wait() is called.
• sleep just suspends a thread execution for a fixed time, but wait suspends a thread execution until notify is called.
• wait must happen in a block synchronized on the monitor object (otherwise occurs IllegalMonitorStateException ) whereas sleep does not.

Object lock = ...;
synchronized (lock) {
    lock.wait();
} 

• You can wait on object itself whereas you call sleep on Thread.

Synchronization

Threads communicate primarily by sharing access to fields and the objects reference fields refer to. This form of communication is extremely efficient, but makes two kinds of errors possible: thread interference and memory consistency errors. The tool needed to prevent these errors is synchronization.

However, synchronization can introduce threads deadlock and thread contention (starvation and livelock).

• Thread Interference: Errors are introduced when multiple threads access shared data.
• Memory Consistency Errors: Errors that result form inconsistent views of shared memory.
• Synchronized Methods: It can effectively prevent thread interference and memory consistency errors.
• Implicit Locks: synchronization is based on implicit locks.
• Atomic Access: operations that can’t be interfered with by other threads.

Thread Interference

class Counter {
    private int c = 0;
    public void increment(){
        c++;
    }
    public void decrement(){
        c--;
    }
    public int value(){
        return c;
    }
}

The increment() and decrement() are not atomic operations. Each method has three steps operations: retrieve variable, update value, store result in variable. When multiple threads invoke these methods, they interfere with each other, because the three steps of each thread are interleaving executed. So the result of each thread is unpredictable.

Memory Consistency Errors

Memory consistency errors occur when different threads have inconsistent views of what should be the same data. When other thread update value of a variable, your current thread still read old value of the variable.

The key to avoiding memory consistency errors is understanding the happens-before relationship. This relationship is simply a guarantee that memory writes by one specific statement is visible to another specific statement.

There are several actions that create happens-before relationships:

• Single thread rule: Each action in a single thread happens-before every action in that thread that comes later in the program order.
• Monitor lock rule (synchronization): An unlock on a monitor lock (exiting synchronized method/block) happens-before every subsequent acquiring on the same monitor lock.
• Volatile variable rule: A write to a volatile field happens-before every subsequent read of that same field.
• Thread start rule: A call to Thread.start() on a thread happens-before every action in the started thread.
• Thread join rule: All actions in a thread happen-before any other thread successfully returns from a join on that thread.
• Transitivity: If A happens-before B, and B happens-before C, then A happens-before C.

Synchronized Methods

The Java programming language provides two basic synchronization idioms: synchronized methods and synchronized statements. synchronized make non-atomic operations become atomic operations and establish happens-before relationships between threads that access the same variables.

class SynchronizedCounter {
    private int c = 0;
    public synchronized void increment(){
        c++;
    }
    public synchronized void decrement(){
        c--;
    }
    public synchronized int value(){
        return c;
    }
}

Synchronized methods enable a simple strategy for preventing thread interference and memory errors. First, it is not possible for two invocations of synchronized methods on the same object to interleave. Second, when a synchronized method exists, it automatically establishes a happens-before relationship with any subsequent invocation of a synchronized method for the same object.

Intrinsic Locks and Synchronization

Synchronization is built around an internal entity known as the intrinsic lock or monitor lock. (The API specification often refers to this entity simply as a “monitor.”) Intrinsic locks play a role in both aspects of synchronization: enforcing exclusive access to an object’s state and establishing happens-before relationships.

Every object has an intrinsic lock associated with it.

Locks in Synchronized Methods

When a thread invokes a synchronized method, it automatically acquires the intrinsic lock for that method’s object and release it when the method returns. The lock release occurs even if the return was caused by an uncaught exception.

A static synchronized method is associated with a class. A thread acquires the intrinsic lock for the Class object associated with the class.

Synchronized Statements

public void add(String name){
    synchronized(this){
        lastName = name;
        nameCount++;
    }
    nameList.add(name);
}

Invoking other objects’ methods from synchronized code can create liveness problems.

Synchronized statements are also useful for improving concurrency with fine-grained synchronization.

Reentrant Synchronization

Allowing a thread to acquire the same lock more than once enable reentrant synchronization.

Atomic Access

Common Atomic Actions

• Reads and writes are atomic for reference variables and for most of primitive variables (except long and double).
• Reads and writes are atomic for all variable declared volatile (including long and double variables)

Atomic actions cannot be interleaved, so they can be used without fear of thread interference. However, this does not eliminate all need to synchronize atomic actions, because memory consistency errors are still possible. Using volatile variables reduce the risk of memory consistency errors, because any write to a volatile variable establishes a happens-before relationship with subsequent reads of the same variable.

Using simple atomic variable access is more efficient than accessing these variables through synchronized code, but requires more care by the programmer to avoid memory consistency errors. Some of the classes in the java.util.concurrent package provide atomic methods that do not rely on synchronization.

Liveness

The most common kind of liveness problem is the deadlock, others are starvation and livelock.

Deadlock

Deadlock describes a situation where two or more threads are blocked forever, waiting for each other.

public class DeadlockExample{
    private static Object lock1 = new Object();
    private static Object lock2 = new Object();

    public static void main(String[] args) {
        new Thread(()->{
            synchronized (lock1){
                try {
                    // ensure both two threads acquired their first lock, 
                    // and waiting for acquired the second lock.
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                synchronized (lock2){}
            }
        }).start();
        new Thread(()->{
            synchronized (lock2){
                try {
                    Thread.sleep(1000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                synchronized (lock1){}
            }
        }).start();
    }
}

Starvation and Livelock

Starvation and livelock are much less common a problem than deadlock, but are still problems that every designer of concurrent software is likely to encounter.

Starvation describes a situation where a thread is unable to gain regular access to shared resources and is unable to make progress.

Livelock is a situation threads simply too busy responding to each other and unable to make further progress.

Guarded Blocks

Threads often have to coordinate their actions. The most common coordination idiom is the guarded block. Such a block begins by polling a condition that must be true before the block can proceed.

Guard by simply loop

public void guardedJoy(){
	while(!joy) {}
	System.out.println("Joy has been achived!");
}

Guard by invokes Object.wait. A more efficient guard.

public synchronized void guardedJoy(){
    while(!joy){
        try{
            wait();
        } catch (InterruptedException e){}
    }
    System.out.println("Joy efficiently has been achived!");
}

public synchronized notifyJoy() {
    joy = true;
    notifyAll();
}

Using guarded blocks can create a Producer-Consumer application.

Immutable Objects

An object is considered immutable if its state connot change after it is constructed.

Immutable objects are particularly useful in concurrent applications. Since they cannot change state, they cannot be corrupted by thread interference or observed in an inconsistent state.

Immutable objects using strategies

• Don’t provide “setter” methods. Methods that modify fields or objects referred to by fields.
• Make all fields final and private.
• Don’t allow subclass to override methods. The simplest way to do this is to declare the class as final. A more sophisticated approach is to make the constructor private and construct instances in factory methods in this class.
• If the instance fields include references to mutable objects, don’t allow those objects to be change. Don’t provide methods that modify the mutable objects. Don’t share references to the mutable objects.

High Level Concurrency Objects

These low-level APIs are adequate for very basic tasks, but higher-level building blocks are needed for more advanced tasks. This is especially useful for massively concurrent applications in multiprocessor and multi-core systems.

High Level Concurrency Features:

• Lock object support locking idioms that simplify many concurrent applications.
• Executors define a high-level API for launching and managing threads. Executor implementations provided by java.util.concurrent provide thread pool management suitable for large-scale applications.
• Concurrent collections make it easier to manage large collections of data, and can greatly reduce the need for synchronization.
• Atomic variables have features that minimize synchronization and help avoid memory consistency errors.
• ThreadLocalRandom provides efficient generation of pseudorandom numbers from multiple threads.

Lock Objects

Synchronized code relies on a simple kind of reentrant lock. This kind of lock is easy to use, but many limitations. More sophisticated locking idioms are supported by the java.util.concurrent.locks package.

Lock objects work very much like the implicit locks used by synchronized code. The biggest advantage of Lock objects over implicit locks is their ability to back out of an attempt to acquire a lock. The tryLock method backs out if the lock is not available immediately or before a timeout expires.

We can use Lock objects to solve the deadlock problem. First we use Lock.tryLock() method to acquire all locks we needed, if fail to acquire, then unlock all acquired locks, else we can acquire all locks and without deadlock problem.

Executors

In large-scale applications, it makes sense to separate thread management and creation from the rest of the application. Objects that encapsulate these functions are known as executors.

• Executor Interfaces: defines the three executor object types.
• Thread Pools: are the most common kind of executor implementation.
• Fork/Join: is a framework for taking advantage of multiple processors.
• Executors: is a utility class that has factory and utility methods for Executor, ExecutorService, ScheduledExecutorService, ThreadFactory, and Callable classes.

Hierarchy of Executor

(I)Executor
|----(I)ExecutorService
|--------(I)ScheduledExecutorService
|--------(A)AbstractExecutorService
|------------ForkJoinPool
|------------ThreadPoolExecutor
|----------------ScheduledThreadPoolExecutor
Executors

Executor Interfaces

The java.util.concurrent package defines three executor interfaces: Executor, ExecutorService, ScheduledExecutorService. Typically, variables that refer to executor objects are declared as one of these three interface types, not with an executor class type.

The Executor interface provides a single method execute() designed to be a replacement for a common thread-creation idiom.

new Thread(r).start();

executor.execute(r);

The ExecutorService interface provides methods execute and submit. Like execute, the submit method accepts Runnable objects, but also accepts Callable object, which allow the task to return a value. The submit method returns a Future object.

The ScheduledExecutorService interface supplements the methods of its parent ExecutorService with schedule, which executes a Runnable or Callable task after a specified delay. In addition, the interface defines scheduleAtFixedRate and scheduleWithFixedDelay, which executes specified tasks repeatedly, at defined intervals.

Thread Pools

Most of executor implementations use thread pools. One common type of thread pool is the fixed thread pool.

One common type of thread pool is the fixed thread pool. A simple way to create an executor that uses a fixed thread pool is to invoke the newFixedThreadPool factory method of java.util.concurrent.Executors. This class also provides the following factory methods:

• newCachedThreadPool
• new SingleThreadExecutor
• several factory methods returns ScheduledExecuteorService executors.

If none of the executors provided by the factory methods of Executors meet your needs, constructing instance of ThreadPoolExecutor or ScheduledThreadPoolExecutor will give you additional options.

To construct a ThreadPoolExecutor object, you least specify five arguments:

• ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue)

Fork/Join

The fork/join framework is an implementation of ExecutorService interface that helps you take advantage of multiple processors. It is designed for work that can be broken into smaller pieces recursively.

The fork/join framework is distinct because it uses a work-stealing algorithm. Worker threads that run out of things to do can steal tasks from other threads that are still busy.

The center of the fork/join framework is the ForkJoinPool class, an extension of the AbstractExecutorService class. ForkJoinPool implements the core work-stealing algorithm and can execute ForkJoinTask processes.

You submit tasks to a ForkJoinPool similarly to how you submit tasks to an ExecutorService. You can submit two types of tasks. One is RecursiveAction that does not return any result. Another is RecursiveTask can return a result. They are a subclass of ForkJoinTask, and all of the tasks classes are abstract.

There are some generally useful features in JavaSE which are already implemented using the fork/join framework.

• Arrays class methods: parallelSort()
• java.util.stream package

fork/join framework pseudocode:

if (my portion of the work is small enough)
	do the work directly
else
	split my work into two pieces

Example of ForkJoinPool

public class Main {
    public static void main(String[] args) {
        ForkJoinPool forkJoinPool = new ForkJoinPool(5);
        forkJoinPool.invoke(new MyRecursiveAction(100));
    }
}

public class MyRecursiveAction extends RecursiveAction {
    private long workload = 0;
    private static final long THRESHOLD = 16;
    
    public MyRecursiveAction(long workload) {
        this.workload = workload;
    }

    @Override
    protected void compute() {
        if (this.workload <= THRESHOLD) {
            System.out.println("Doing workload myself : " + this.workload);
        } else {
            System.out.println("Splitting workload : " + this.workload);
            List<MyRecursiveAction> subtasks = new ArrayList<>();
            subtasks.addAll(createSubtasks());
            for (RecursiveAction subtask : subtasks) {
                subtask.fork();
            }
        }
    }

    private List<MyRecursiveAction> createSubtasks() {
        List<MyRecursiveAction> subtasks = new ArrayList<>();
        MyRecursiveAction subtask1 = new MyRecursiveAction(this.workload / 2);
        MyRecursiveAction subtask2 = new MyRecursiveAction(this.workload / 2);
        subtasks.add(subtask1);
        subtasks.add(subtask2);
        return subtasks;
    }
}

Concurrent Collections

Concurrent collections are high concurrency and thread-safe collection implementation. For example, CopyOnWriteArrayList, ConcurrentHashMap.

Atomic Variables

The java.util.concurrent.atomic package defines classes that support atomic operations on single variable. All classes have get and set methods that work like reads and writes on volatile variables. A set has a happens-before relationship with any subsequent get on the same variable. The atomic compareAndSet method also has these memory consistency feature, as do the simple atomic arithmetic methods that apply to integer atomic variables.

Example of Basic Usage of AtomicInteger

AtomicInteger atomicInteger = new AtomicInteger(0);
atomicInteger.accumulateAndGet(10, (x, y)->{return x + y;});
System.out.println(atomicInteger.get());

Example of Using Atomic Variable instead of Synchronized Methods

public class AtomicCounter {
    private AtomicInteger c = new AtomicInteger(0);

    public void increment() {
        c.incrementAndGet();
    }

    public void decrement() {
        c.decrementAndGet();
    }

    public int value() {
        return c.get();
    }
}

Concurrent Random Numbers

The package java.util.concurrent includes a convenience class, ThreadLocalRandom, for applications that expect to use random numbers from multiple threads or ForkJoinTasks.

For concurrent access, using ThreadLocalRandom instead of Math.random() results in less contention and better performance.

Example of ThreadLocalRandom

int r = ThreadLocalRandom.current().nextInt(1, 10);

Conclusion

In multi-threads programming, allows multiple threads simultaneously access shared resources that may cause unpredicted results or corrupted values. For thread-safe, we need to consider two factors: thread interference and memory consistency errors.

Thread interference can be solved by atomic access (common using exclusive locks). Memory consistency errors can be solved by establishing a happens-before relationship (that between reads and writes the same variable).

We can simply use synchronized to solve the two thread-safe problems. But for high concurrency and thread-safe, we should use as few locks as possible, even have no locks. So we consider using a combination of explicit locks (reentrant locks), immutable objects, volatile variables, and atomic variables to solve the two thread-safe problems.

Locks may cause liveness problems: deadlock, starvation, and livelock. we can follow some coding principles to avoid these problems happens.

Guarded blocks common use for in threads cooperation. The most common example is the Producer-Consumer application.

Executors are for efficient thread creation and management.

Concurrent collections are high concurrency and thread-safe collection implementation.

Understanding Java articles

• Understanding Java：IO
• Understanding Java：Lambda and Stream
• Understanding Java：Container
• Understanding Java：Concurrency
• Understanding Java：Proxy
• Understanding Java：NIO

References

[1] The Java Tutorial: A Short Course on the Basics

[2] Difference between wait() and sleep()

[3] What does java.lang.Thread.interrupt() do?

[4] Java - Understanding Happens-before relationship

[5] Memory Consistency - happens-before relationship in Java

This post will discuss content in Java container. It’s based Java SE 8.

Collection

Collection interface

The root interface in collection hierarchy. A collection represents a group of objects, known as its elements. Some collections allow duplicate elements and others do not. some are ordered and others unordered. The JDK does not provide any direct implementations of this interface. This interface is typically used to pass collections around and manipulate them where maximum generality is desired.

Methods of Collection

Basic Operations

• boolean add(E e), boolean addAll(Collection<? extends E> c)
• boolean remove(Object o), boolean removeAll(Collection<?> c)
• void clear()
• boolean contains(Object o), boolean contains(Collection<?> c)
• int size()
• boolean equals(Object o)
• boolean isEmpty()
• Iterator<E> iterator()
• object[] toArray(), <T> T[] toArray(T[] a)

Advanced Operations

• default Stream<E> stream()
• default Stream<E> parallelStream()
• default boolean removeIf(Predicate<? super E> filter)
• boolean retainAll(Collection<?> c)
• default Spliterator<E> spliterator()

AbstractCollection

This class provides a skeletal implementation of the Collection interface, to minimize the effort required to implement Collection interface.

Most of methods implementation of AbstractCollection depend on its abstract methods iterator() and size(). These two methods are the most important and useful for collection classes.

Non-abstract methods in this class may be overridden if its subclass implemented admits a more efficient implementation.

List

Hierarchy of List:

(I)List
|----(A)AbstractList
|--------ArrayList
|--------Vector
|--------(A)AbstractSequentialList
|------------LinkedList
|----java.util.concurrent.CopyOnWriteArrayList

List interface

This interface control over where to insert. The user can access elements by their integer index, and search for elements in the list.

The List interface provides a special iterator, called a ListIterator, that allows element insertion add(E e) and replacement set(E e), and bidirectional access previous() in addition to the normal operations that the Iterator interface provides.

Additional methods of List

• E get(int index)
• int indexOf(Object o)
• int lastIndexOf(Object o)
• ListIterator listIterator()
• void replaceAll(UnaryOperator operator)
• E set(int index, E element)
• void sort(Comparator c)
• List<E> sublist(int fromIndex, int toIndex)

AbstractList

This class provides a skeletal implementation of the List interface to minimize the effort required to implement this interface backed by a “random access” data store (such as an array).

AbstractSequentialList

This class provides a skeletal implementation of the List interface to minimize the effort required to implement this interface backed by a “sequential access” data store (such as a linked list).

ArrayList & Vector

Resizable-array implementation of the List interface. Implements all optional list operations, and permits all elements, including null. In addition to implementing the List interface, this class provides methods to manipulate the size of the array that is used internally to store the list.

ArrayList class is roughly equivalent to Vector, except that it is unsynchronized.

Each ArrayList instance has a capacity. The capacity is the size of the array used to store the elements in the list. It is always at least as large as the list size. As elements are added to an ArrayList, its capacity grows automatically. The details of the growth policy are not specified beyond the fact that adding an element has constant amortized time cost.

An application can increase the capacity of an ArrayList instance before adding a large number of elements using the ensureCapacity operation. This may reduce the amount of incremental reallocation.

Note that this implementation is not synchronized. If multiple threads access an ArrayList instance concurrently, and at least one of the threads modifies the list structurally, it must be synchronized externally. If the list used in multithread, the list should be “wrapped” using the Collections.synchronizedList method. This is best done at creation time, to prevent accidental unsynchronized access to the list.

The iterators returned by this class’s iterator and listIterator methods are fail-fast: if the list is structurally modified at any time after the iterator is created, in any way except through the iterator’s own remove or add methods, the iterator will throw a ConcurrentModificationException. Thus, in the face of concurrent modification, the iterator fails quickly and cleanly, rather than risking arbitrary, non-deterministic behavior at an undetermined time in the future.

LinkedList

Doubly-linked list implementation of the List and Deque interfaces.

All of the operations perform as could be expected for a doubly-linked list. Operations that index into the list will traverse the list from the beginning or the end, whichever is closer to the specified index.

Note that this implementation is not synchronized. If multiple threads access a linked list concurrently, and at least one of the threads modifies the list structurally, it must be synchronized externally. If the list used in multithread, the list should be “wrapped” using the Collections.synchronizedList method. This is best done at creation time, to prevent accidental unsynchronized access to the list.

The iterators returned by this class’s iterator and listIterator methods are fail-fast. Same with ArrayList.

CopyOnWriteArrayList

What is CopyOnWriteArrayList

java.util.concurrent.CopyOnWriteArrayList a thread-safe variant of ArrayList in which all mutative operation (add, set, and so on) are implemented by making a fresh copy of the underlying array.

This is ordinarily too costly, but may be more efficient than alternatives when traversal operations vastly outnumber mutations, and is useful when you cannot or don’t want to synchronize traversals, yet need to preclude interference among concurrent threads. The “snapshot” style iterator method uses a reference to the state of the array at the point that the iterator was created. This array never changes during the lifetime of the iterator, so interference is impossible and the iterator is guaranteed not to throw ConcurrentModificationException. The iterator will not reflect additions, removals, or changes to the list since the iterator was created. Element-changing operations on iterators themselves (remove, set, and add) are not supported. These methods throw UnsupportedOperationException.

Why is the CopyOnWriteArrayList Thread-safe

All write operations have a lock. The write operations are serializable.

All mutative operations operate on new copy list and then update volatile list reference. Array update is an atomic operation and hence reads will always see the array in a consistent state.

Applicability

Updating this collection a lot will kill performance. You should only use this read when you are doing upwards of 90+% reads.

Consequences

The all read and write operations of CopyOnWriteArrayList are thread-safe. Some of methods have a ReentrantLock and copy list, some have not.

It highly improves concurrency of read operations in thread-safe condition, but write operations is very costly.

Questions

Q: Why all mutative operations need to copy list? Why read and traversal operation not need locks?

A: The key to this idea is that writes are accomplished by copying the existing value, modifying it, and replacing the reference. It also follows that once set the object pointed by the reference is always read only (i.e. no mutation is done directly on the object referred by the reference). Therefore, readers can access it safely without synchronization.

Queue

Hierarchy of Queue:

(I)java.util.Queue
|----(A)java.util.AbstractQueue
|--------java.util.PriorityQueue
|----(I)java.util.Deque
|--------java.util.ArrayDeque
|----(I)java.util.concurent.BlockingQueue
|--------(I)BlockingDeque
|------------LinkedBlockingDeque
|--------(I)TransferQueue
|------------LinkedTransferQueue
|--------ArrayBlockingQueue
|--------LinkedBlockingQueue
|--------PriorityBlockingQueue
|--------DelayQueue
|--------SynchronousQueue
|----java.util.concurrent.ConcurrentLinkedQueue
|----java.util.concurrent.ConcurrentLinkedDeque

Queue interface

A group of elements in First in first out manner.

The Queue interface does not define the blocking queue methods, which are common in concurrent programming.

Queue implementations generally do not allow insertion of null elements, although some implementations do not prohibit. Null should not be inserted into a queue, as null is also used as a special return value by the poll method to indicate that the queue contains no elements.

Methods of Queue

• insert
  • boolean add(E e) - Inserts element into this queue.
  • boolean offer(E e) - It is similar with add().
• remove
  • E poll() - Retrieves and remove the head of this queue.
  • E reomve() - It is similar with poll()
• examine
  • E element() - It is similar with peek().
  • E peek() - Retrieves, but does not remove, the head of this queue.

Deque interface

A group of elements supports element insertion and removal at both head and tail. Deque can also be used as LIFO (last-in-First-out) stacks.

The name deque is short for “double ended queue” and is usually pronounced “deck”.

Methods of Deque

• insert
  • boolean add(E e), void addFirst(E e), void addLast(E e)
  • boolean offer(E e), boolean offerFirst(E e), boolean offerLast(E e)
  • void push(E e)
• remove
  • E remove(), E removeFirst(), E removeLast()
  • boolean removeFirstOccurrence(Object o), boolean removeLastOccurrence(Object o)
  • E poll(), E pollFirst(), E pollLast()
  • E pop()
• examine
  • E getFirst(), E getLast()
  • E peek(), E peekFirst(), E peekLast()
  • E element()
• boolean contains(Object o)
• Iterator descendingIterator()
• Iterator iterator()
• int size()

BlockingQueue interface

java.util.concurrent.BlockingQueue is a queue additionally supports operations that wait for the queue to become non-empty when retrieve an element, and wait for space to become available in the queue when storing an element.

BlockingQueue implementations are designed to be used primarily for producer-consumer queues. A BlockingQueue can safely be used with multiple producers and multiple consumers.

BlockingQueue implementations are thread-safe. All queuing methods achieve their effects atomically using internal locks or other forms of concurrency control. However, the bulk Collection operations addAll, containsAll, retainAll and removeAll are not necessarily performed atomically. So it is possible, for example, for addAll(c) to fail after adding only some of the elements in c.

Methods of BlockingQueue

• insert
  • boolean add(E e)
  • boolean offer(E e)
  • void put(E e) - blocks
  • boolean offer(E e, long time, TimeUnit unit) - blocks and times out
• remove
  • boolean remove(Object o)
  • E poll()
  • E take() - blocks
  • E poll(long time, TimeUnit unit) - blocks and times out
• examine
  • E element()
  • E peek()
• boolean contains(Object o)
• drainTO(Collection c)
• int remainingCapacity()

Usage example of producer-consumer

class Producer implements Runnable {
   private final BlockingQueue queue;
   Producer(BlockingQueue q) { queue = q; }
   public void run() {
     try {
       while (true) { queue.put(produce()); }
     } catch (InterruptedException ex) { ... handle ...}
   }
   Object produce() { ... }
 }

 class Consumer implements Runnable {
   private final BlockingQueue queue;
   Consumer(BlockingQueue q) { queue = q; }
   public void run() {
     try {
       while (true) { consume(queue.take()); }
     } catch (InterruptedException ex) { ... handle ...}
   }
   void consume(Object x) { ... }
 }

 class Setup {
   void main() {
     BlockingQueue q = new SomeQueueImplementation();
     Producer p = new Producer(q);
     Consumer c1 = new Consumer(q);
     Consumer c2 = new Consumer(q);
     new Thread(p).start();
     new Thread(c1).start();
     new Thread(c2).start();
   }
 }

BlockingDeque interface

It is the combination of BlockingQueue and Deque interface. You can use this interface for both blocking queues and blocking stacks.

TransferQueue interface

The TransferQueue interface is a refinement of the BlockingQueue interface in which producers can wait for consumers to receive elements. The BlockingQueue can only put element into queue (and block if queue is full). However, with TransferQueue, you can also block producer (whether the queue full or not) until other consumer threads receive your element.

Methods of TransferQueue

• int getWaitingConsumerCount() - Returns an estimate of the number of consumers waiting to receive elements via take() or timed poll.
• boolean hasWaitingConsumer() - Returns true if there is at least one consumer waiting to receive an element via take() or timed poll.
• void transfer(E e) - Transfers the element to a consumer, waiting if necessary to do so.
• boolean tryTransfer(E e) - Transfers the element to a waiting consumer immediately, if possible.
• boolean tryTransfer(E e, long timeout, TimeUnit unit) - Transfers the element to a consumer if it is possible to do so before the timeout elapses.

Usage Example of TransferQueue

public class TransferQueueExample {

    TransferQueue<String> queue = new LinkedTransferQueue<String>();

    class Producer implements Runnable{
        @Override
        public void run() {
            for(int i = 0; i < 2; i++){
                try{
                    System.out.println("Producer waiting to transfer: " + i);
                    queue.transfer("" + i);
                    System.out.println("Producer transfered: " + i);
                }catch(Exception e){
                    e.printStackTrace();
                }
            }
        }

    }

    class Consumer implements Runnable{
        @Override
        public void run() {
            for(int i = 0; i < 2; i++){
                try{
                    Thread.sleep(2000);
                    System.out.println("Consumer waiting to comsume: " + i);
                    queue.take();
                    System.out.println("Consumer consumed: " + i);
                }catch(Exception e){
                    e.printStackTrace();
                }
            }
        }
    }

    public static void main(String args[]){
        TransferQueueExample example = new TransferQueueExample();
        new Thread(example.new Producer()).start();
        new Thread(example.new Consumer()).start();
    }
}

The output is:

Producer waiting to transfer: 0
Consumer waiting to comsume: 0
Consumer consumed: 0
Producer transfered: 0
Producer waiting to transfer: 1
Consumer waiting to comsume: 1
Consumer consumed: 1
Producer transfered: 1

Producer thread is blocked before consumer thread take product away.

AbstractQueue

This class provides skeletal implementations of some Queue operations. Its implemented methods actually implementing by calling abstract methods offer(), poll(), peek().

PriorityQueue

An unbounded priority queue based on a priority heap. The elements of the priority queue are ordered according to their natural Order (ascending order. sort by e1.compareTo(e2)), or by a Comparator provided at queue construction time, depending on which constructor is used. A priority queue does not permit null elements. A priority queue relying on natural Order also does not permit insertion of non-comparable objects.

Its implementation provides O(log(n)) time for the enqueuing and dequeuing methods (offer, poll, remove() and add); linear time for the remove(Object) and contains(Object) methods; and constant time for the retrieval methods (peek, element, and size).

Concurrent Queues

DelayQueue

An unbounded blocking queue of Delayed elements, in which an element can only be taken when its delay has expired. If no delay has expired there is no head and poll will return null.

DelayQueue<E extends Delay> is a PriorityQueue order by Delayed.

Methods of Delayed interface

• long getDelay(TimeUnit unit) - Returns the remaining delay associated with this object, in the given time unit.
• int compareTo(Delayed obj)

Methods of DelayQueue similar with BlockingQueue.

Usage Example

class MyDelayed implements Delayed {
    private String name;
    private long time;

    public MyDelayed(String name, long delayMillisecond) {
        this.name = name;
        this.time = System.currentTimeMillis() + delayMillisecond;
    }

    @Override
    public long getDelay(TimeUnit unit) {
        return unit.convert(time - System.currentTimeMillis(), TimeUnit.MILLISECONDS);
    }

    @Override
    public int compareTo(Delayed o) {
        return (int) (this.getDelay(TimeUnit.MILLISECONDS) - o.getDelay(TimeUnit.MILLISECONDS));
    }
    
    @Override
    public String toString() {
        return "MyDelayed{" + "name='" + name + '\'' + ", time=" + time + '}';
    }
}

public class Test{
    public static void main(String[] args) throws InterruptedException {
        DelayQueue<MyDelayed> delayQueue = new DelayQueue();
        // offer
        delayQueue.offer(new MyDelayed("B", 3000L));
        delayQueue.offer(new MyDelayed("A", 2L));
        // poll
        // waiting for the delay is expired to poll
        Thread.sleep(2L);
        System.out.println("before poll");
        // Output: MyDelayed{name='A', time=1587625828209}
        System.out.println(delayQueue.poll()); 
        // Output is null, because the delay is not expired
        System.out.println(delayQueue.poll());
    }
}

SynchronousQueue

A blocking queue in which each insert operation must wait for a corresponding remove operation by another thread, and vice verse. A synchronous queue does not have any internal capacity.

The insert and remove operations of SynchronousQueue may block its thread. The insert and remove operations must occur at the same time, and the fast thread will block to wait for another operation thread.

public static void main(String[] args) throws InterruptedException {
    SynchronousQueue<Integer> synchronousQueue = new SynchronousQueue<>();
    new Thread(()->{
        try {
            Thread.sleep(1000L);
            System.out.println("the put to wait for remove...");
            synchronousQueue.put(1);
            System.out.println("end to put.");
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }).start();
    new Thread(()->{
        try {
            System.out.println("the remove to wait for put...");
            System.out.println(synchronousQueue.take());
            System.out.println("end to remove.");
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }).start();
}

Output

the remove to wait for put...
the put to wait for remove...
end put.
1
end remove.

ConcurrentLinkedQueue

An unbound thread-safe queue based on linked nodes. This queue orders elements FIFO (first-in-first-out). A ConcurrentLinkedQueue is an appropriate choice when many threads will share access to a common collection.

Stack

Hierarchy of Stack:

java.util.Stack
java.util.Deque
java.util.LinkedList
java.util.ArrayList

java.util.Stack class represents a last-in-first-out sequence. It extends class Vector. It’s thread-safe implemented by synchronized methods.

If you don’t operate in a multithread environment, you should use Deque as Stack that is fast than Stack.

Methods of Stack

• E peek()
• E pop()
• E push(E item)
• boolean empty()
• int search(Object o)

Set

Hierarchy of Set:

(I)Set
|----(A)AbstractSet
|--------HashSet
|------------LinkedHashSet
|--------(A)EnumSet
|--------java.util.concurrent.CopyOnWriteArraySet
|----(I)SortedSet
|--------(I)NavigableSet
|------------TreeSet
|------------java.util.concurrent.ConcurrentSkipListSet

Set Interface

A collection that contains no duplicate elements, and at most one null element.

Methods of Set interface

• boolean add(E e), addAll(Collection c)
• void clear()
• boolean contains(Object o), containsALl(Collection c)
• boolean equals(Object o)
• int hashCode()
• boolean isEmpty()
• Iterator iterator()
• boolean remove(Object o), removeAll(Collection c)
• boolean retainAll(Collection c)
• int size()
• default Spliterator spliterator()
• Object[] toArray(), T[] toArray(T[] a)

The Set interface does not have any get or find methods to find an element from the Set container. The only way to find the specified element is by using its iterator to traverse the Set.

AbstractSet Class

This class provides a skeletal implementation of the Set interface to minimize the effort required to implement this interface.

Implemented methods of Set interface:

• equals(Object o)
• hashCode()
• removeAll(Collection c)

HashSet Class

This class implements the Set interface, backed by a hash table (actually a HashMap instance). It makes no guarantees as to the iteration order of the set.

This class offers constant time performance for the basic operation (add, remove, contains and size), assuming the hash function disperses the elements properly among the buckets.

This implementation is not synchronized.

This class’s iterator is fail-fast.

LinkedHashSet

Hash table and linked list implementation of Set interface, with predictable iteration order. This implementation differs from HashSet in that it maintains a doubly-linked list running through all of its entries. This linked list defines the iteration Order, which is insertion-order.

EnumSet Abstract Class

A specialized Set implementation for use with enum types. All of the elements in an enum set must come from a single enum type. Enum sets are represented internally as bit vectors. This representation is extremely compact and efficient.

SortedSet Interface

SortedSet is a Set that provides a total Order on its elements. The elements are ordered using their natural Order, or by a Comparator typically provided at sorted set creation time. The set’s iterator will traverse the set in ascending element order.

NavigableSet interface

NavigableSet is a SortedSet textended with navigation methods reporting closest matches for given search targets. Methods lower, floor, ceiling, and higher return elements respectively less than, less than or equal, greater than or equal, and greater than a given element, returning null if there is no such element. A NavigableSet may be accessed and traversed in either ascending or descending order.

Methods of NavigableSet interface

• Navigation
  • E ceiling(E e)
  • E floor(E e)
  • E higher(E e)
  • E lower(E e)
  • E pollFirst()
  • E pollLast()
• Iterator
  • Iterator descendingIterator()‘
  • Iterator iterator()
• Subset
  • SortedSet headSet(E toElement), NavigableSet headSet(E toElement, boolean inclusive)
  • SortedSet tailSet(E fromElement), NavigableSet tailSet(E fromElement, boolean inclusive)
  • SortedSet subSet(E fromElement, E toElement), NavigableSet subSet(E fromElement, boolean fromInclusive, E toElement, boolean toInclusive)
• Reverse
  • NavigableSet descendingSet()

TreeSet Class

TreeSet is a NavigableSet implementation based on TreeMap. The elements are ordered using their natural ordering, or by a Comparator provided at set creation time, depending on which constructor is used.

This implementation provides guaranteed log(n) time cost for the basic operations (add, remove, and contains).

This implementation is not synchronized. If you want to use TreeSet in multithread environment, you can using the Collections.synchronizedSortedSet(new TreeSet()) to wrap it.

Concurrent Set

CopyOnWriteArraySet

CopyOnWriteArraySet is a Set that uses an internal CopyOnWriteArrayList for all of its operations.

It shares basic properties:

• It is best suited for applications in which set sizes generally stay small, read-only operations vastly outnumber mutative operations, and you need to prevent interference among threads during traversal.
• It is thread-safe.
• Mutative operations (add, set, remove, etc.) are expensive since they usually entail copying the entire underlying array.
• Iterators do not support the mutative remove operation.
• Traversal via iterators is fast and cannot encounter interference from other threads. Iterators rely on unchanging snapshots of the array at the time the iterators were constructed.

ConcurrentSkipListSet

ConcurrentSkipListSet is a scalable concurrent NavigableSet implementation based on a ConcurrentSkipListMap. The elements of the set are kept sorted according to their natural ordering, or by a Comparator provided at set creation time, depending on which constructor is used.

This implementation provides expected average log(n) time cost for the contains, add, and remove operations and their variants. Insertion, removal, and access operations safely execute concurrently by multiple threads.

Iterators and spliterators are weakly consistent.

Beware that, unlike in most collections, the size method is not a constant-time operation. Because of the asynchronous nature of these sets, determining the current number of elements requires a traversal of the elements, and so may report inaccurate results if this collection is modified during traversal. Additionally, the bulk operations addAll, removeAll, retainAll, containsAll, equals, and toArray are not guaranteed to be performed atomically. For example, an iterator operating concurrently with an addAll operation might view only some of the added elements.

Map

Hierarchy of Map:

(I)Map
|----Hashtable
|----(A)AbstractMap
|--------HashMap
|------------LinkedHashMap
|--------EnumMap
|--------IdentityHashMap
|--------WeakHashMap
|----(I)SortedMap
|--------(I)NavigableMap
|------------TreeMap
|----(I)java.util.concurrent.ConcurrentMap
|--------ConcurrentHashMap
|--------(I)ConcurrentNavigableMap
|------------ConcurrentSkipListMap

Map interface

An object that maps keys to values. A map connot contain duplicate keys, and each key can map to at most one value.

The Map interface provides three collection views, a set of keys, collection of values, or set of key-value mappings.

Methods of Map interface

• Basic

  • V get(Object key)
  • V put(K key, V value)
  • V remove(Object key)
  • int size()
  • boolean isEmpty()
  • void clear()
  • boolean containsKey(Object key)
  • boolean containsValue(Object value)
  • void putAll(Map m)
  • default V putIfAbsent(K key, V value)
  • default V getOrDefault(Object key, V defaultValue)
  • default boolean remove(Object key, Object value)
  • default V replace(K key, V value)
  • default boolean replace(K key, V oldValue, V newValue)

• View

  • Set entrySet()
  • Set keySet()
  • Collection values()

• Function

  • default V compute(K key, BiFunction remappingFunction) - Attempts to compute a mapping for the specified key and its current mapped value. computeIfAbsent(...), computeIfPresent(...)
  • default void forEach(BiConsumer action)

• default V merge(K key, V value, BiFunction remappingFunction)

  • default void replaceAll(BiFunction function)

Note that map means mapping keys to values, it does not specify how to mapping. The hash function is one of the ways to map keys to values.

HashTable

This class implements a hash table, which maps keys to values.

Hashtable is synchronized. If a thread-safe implementation is not needed, it is recommended to use HashMap in place of Hashtable. If a thread-safe highly-concurrent implementation is desired, then it is recommended to use ConcurrentHashMap in place of Hashtable.

AbstractMap

This class provides a skeletal implementation of the Map interface, to minimize the effort required to implement this interface.

It simply implemented some methods of Map just using iterator, like get(), containsKey().

HashMap

Hash table based implementation of the Map interface. It permits null values and null key. This class makes no guarantees order of the map. The HashMap class is roughly equivalent to HashTable, except that it is unsynchronized and permit nulls.

This implementation is not synchronized. If want to use a HashMap object in multithread environment, you can using Collections.synchronizeMap() to wrap it.

EnumMap

A specialized Map implementation for use with enum type keys. All of the keys in an enum map must come from a single enum type that is specified, explicitly or implicitly, when the map is created.

Enum maps are maintained in the natural order of their keys (the order in which the enum constants are declared). This is reflected in the iterators returned by the collections views (keySet(), entrySet(), and values()).

Iterators returned by the collection views are weakly consistent: they will never throw ConcurrentModificationException and they may or may not show the effects of any modifications to the map that occur while the iteration is in progress.

Null keys are not permitted. Attempts to insert a null key will throw NullPointerException.

Like most collection implementations EnumMap is not synchronized. If want to use a EnumMap object in multithread environment, you can using Collections.synchronizeMap() to wrap it.

IdentityHashMap

This class implements the Map interface with a hash table, using reference-equality in place of object-equality when comparing keys (and values). In other words, in an IdentityHashMap, two keys k1 and k2 are considered equal if and only if (k1==k2). For example, the key new String(“a”) and key “a” will different key in IdentityHashMap, but it’s same key in HashMap.

This class is not a general-purpose Map implementation! While this class implements the Map interface, it intentionally violates Map's general contract, which mandates the use of the equals method when comparing objects. This class is designed for use only in the rare cases wherein reference-equality semantics are required.

This implementation is not synchronized. If want to use a IdentityHashMap object in multithread environment, you can using Collections.synchronizeMap() to wrap it.

WeakHashMap

Hash table based implementation of the Map interface, with weak keys. An entry in a WeakHashMap will automatically be removed when its key is no longer in ordinary use. More precisely, the presence of a mapping for a given key will not prevent the key from being discarded by the garbage collector, that is, made finalizable, finalized, and then reclaimed. When a key has been discarded its entry is effectively removed from the map, so this class behaves somewhat differently from other Map implementations.

Each key object in a WeakHashMap is stored indirectly as the referent of a weak reference. Therefore a key will automatically be removed only after the weak references to it, both inside and outside of the map, have been cleared by the garbage collector.

Like most collection classes, this class is not synchronized. A synchronized WeakHashMap may be constructed using the Collections.synchronizedMap method.

Applicability

When you want to automatically remove useless objects from map, you can use WeakHashMap. You can manually remove elements of any types of map, but if don’t or you forget it. It may cause a memory leak.

How does the WeakHashMap Implement?

java.lang.ref.WeakReference

SortedMap interface

A Map that further provides a total ordering on its keys. The map is ordered according to the natural ordering of its keys, or by a Comparator typically provided at sorted map creation time. This order is reflected when iterating over the sorted map’s collection views (returned by the entrySet, keySet and values methods). Several additional operations are provided to take advantage of the ordering.

NavigableMap interface

A SortedMap extended with navigation methods returning the closest matches for given search targets. Methods lowerEntry, floorEntry, ceilingEntry, and higherEntry return Map.Entry objects associated with keys respectively less than, less than or equal, greater than or equal, and greater than a given key, returning null if there is no such key. Similarly, methods lowerKey, floorKey, ceilingKey, and higherKey return only the associated keys. All of these methods are designed for locating, not traversing entries.

TreeMap

A Red-Black tree based NavigableMap implementation. The map is sorted according to the natural ordering of its keys, or by a Comparator provided at map creation time, depending on which constructor is used.

This implementation provides guaranteed log(n) time cost for the containsKey, get, put and remove operations.

This class is not synchronized. A synchronized TreeMap may be constructed using the Collections.synchronizedMap method.

Concurrent Maps

ConcurrentHashMap

What is it

A concurrent map implements by hash table, and supports full concurrency of retrievals and high concurrency for updates. It likes Hashtable, but it has more concurrency.

A hash table supporting full concurrency of retrievals and high expected concurrency for updates. This class obeys the same functional specification as Hashtable, and includes versions of methods corresponding to each method of Hashtable. However, even though all operations are thread-safe, retrieval operations do not entail locking, and there is not any support for locking the entire table in a way that prevents all access. This class is fully interoperable with Hashtable in programs that rely on its thread safety but not on its synchronization details.

ConcurrentSkipListMap

A scalable concurrent ConcurrentNavigableMap implementation. The map is sorted according to the natural ordering of its keys, or by a Comparator provided at map creation time, depending on which constructor is used.

Utility Classes

Collections

What Is It

This class consists of static methods that operate on or return collections.

Methods of Collections

• Operations
  • static boolean addAll(Collection c, T... elements)
  • static int binarySearch(List list, T key), binarySearch(List list, T key, Comparator c)
  • static void copy(List dest, List src)
  • static boolean disjoint(Collection c1, Collection c2)
  • static void fill(List list, T obj)
  • static int frequency(Collection c, Object o)
  • static int indexOfSubList(List source, List target), int lastIndexOfSubList(List source, List target)
  • static T max(Collection c), max(Collection coll, Comparator comp)
  • static T min(Collection c), min(Collection coll, Comparator comp)
  • static boolean replaceAll(List list, T oldVal, T newVal)
  • static void reverse(List list)
  • static Comparator reverseOrder(), reverseOrder(Comparator comp)
  • static void rotate(List list, int distance)
  • static void shuffle(List list), shuffle(List list, Random rnd)
  • static void sort(List list), sort(List list, Comparator comp)
  • static void swap(List list, int i, int j)
• Transforms
  • static Queue asLifoQueue(Deque deque)
  • static Collection checkedCollection(Collection c, Class type), checkedList(List list, Class type), checkedMap, checkedNavigableMap, …
  • static Enumeration enumeration(Collecdtion c)
  • static ArrayList List(Enumeration e)
  • static newSetFromMap(Map map)
  • static Collection synchronizedCollection(Collection c), synchronizedList(List list), synchronizedMap(Map map), synchronizedSet(Set set), …
  • static Collection unmodifiableCollection(Collection), unmodifiableList(List list), unmodifiableMap(Map map), …
• Creations
  • static List emptyList(), emptyMap(), emptySet(), …
  • static List nCopies(int n, T o)
  • static Set singleton(T o), singletonList(T o), singletonMap(K key, V value)

Arrays

What Is It

This class contains various static methods for manipulating arrays (such as sorting and searching). This class also contains a static factory that allows arrays to be viewed as lists.

Methods of Arrays

• Manipulations
  • static int binarySearch(byte[] a, byte key), binarySearch(byte[] a, int fromIndex, int toIndex, byte key), binarySearch(char[] a, char key), binarySearch(int[] a, int key), …
  • static boolean deepEquals(Object[] a1, object[] a2)
  • static int deepHashcode(Object[] a)
  • static String deepToString(Object[] a)
  • static boolean equals(int[] a, int[] a2), equals(byte[] a, byte[] a2), …
  • static void fill(int[] a, int val), fill(byte[] a, byte val), …
  • static int hashcode(int[] a), hashcode(byte[] a), …
  • static void parallelPrefix(int[] array, IntBinaryOperator op), …
  • static void setAll(int[] a, IntUnaryOperator generator), …
  • static void parallelSetAll(int[] a, IntUnaryOperator generator), …
  • static void sort(int[] a), sort(int[] a, int fromIndex, int toIndex), …
  • static void parallelSort(int[] a), parallelSort(int[] a, int fromIndex, int toIndex), parallelSort(byte[] a)
  • static Spliterator spliterator(int[] a), …
  • static Stream stream(int[] a), …
  • static toString(int[] a), …
• Transforms
  • static int[] copyOf(int[] original, int newLength), copyOf(byte[] original, int newLength), …
  • static int[] copyOfRange(int[] original, int from, int to), copyOfRange(byte[] original, int from, int to), …
• Creations
  • static List asList(T... a)

Container Features

Traversal

Fail Fast

when one thread using iterator for traversal, updating the container object by add, remove operations, the iterator thread will throw a ConcurrentModificationException.

Order

Natural Order

Elements order by method compareTo() of the class.

Consistency

Happens-before

This relationship is simply a guarantee that memory writes by one specific statement are visible to another specific statement in multiple threads.

For example, an variable int counter=0 shared in thread A and thread B. If thread A increment counter by counter++, thread B print counter by System.out.println(counter), the printed value would be “1”.

Concurrent Collections

Besides Queues, this package supplies Collection implementations designed for use in multithreaded contexts: ConcurrentHashMap, ConcurrentSkipListMap, ConcurrentSkipListSet, CopyOnWriteArrayList, and CopyOnWriteArraySet. When many threads are expected to access a given collection, a ConcurrentHashMap is normally preferable to a synchronized HashMap, and a ConcurrentSkipListMap is normally preferable to a synchronized TreeMap. A CopyOnWriteArrayList is preferable to a synchronized ArrayList when the expected number of reads and traversals greatly outnumber the number of updates to a list.

The “Concurrent” prefix used with some classes in this package is a shorthand indicating several differences from similar “synchronized” classes. For example java.util.Hashtable and Collections.synchronizedMap(new HashMap()) are synchronized. But ConcurrentHashMap is “concurrent”. A concurrent collection is thread-safe, but not governed by a single exclusion lock. In the particular case of ConcurrentHashMap, it safely permits any number of concurrent reads as well as a tunable number of concurrent writes. “Synchronized” classes can be useful when you need to prevent all access to a collection via a single lock, at the expense of poorer scalability. In other cases in which multiple threads are expected to access a common collection, “concurrent” versions are normally preferable. And unsynchronized collections are preferable when either collections are unshared, or are accessible only when holding other locks.

Most concurrent Collection implementations (including most Queues) also differ from the usual java.util conventions in that their Iterators and Spliterators provide weakly consistent rather than fast-fail traversal:

• they may proceed concurrently with other operations
• they will never throw ConcurrentModificationException
• they are guaranteed to traverse elements as they existed upon construction exactly once, and may (but are not guaranteed to) reflect any modifications subsequent to construction.

Summary

Container Class Contrast Table

List

Queue & Stack

Set

Map

Container Classes core features

• bounded, or unbounded
• ordered, or disordered (insert Order, natural Order, comparator Order)
• null value
• traverse operation is fail-fast or weakly consistent
• performance
  • read, write and traversal operations time and space cost.
• thread safe: concurrency, consistency
  • synchronized
  • concurrent
  • consistent

How to select a effective container class to use

List

• ArrayList and LinkedList are common lists. ArrayList implemented by array, it is fast to random access, but slow to insert and remove. LinkedList implemented by nodes, it is fast to insert and remove, but slow to random access.
• Vector is thread-safe list by synchronized. If you want to use list in multithread environment, you can choose it.
• CopyOnWriteArrayList is also thread-safe list, and It is more concurrent than synchronized, but its write operations is very costly. If you want to use list in multithread environment with high concurrent, and operations are almost read operations(90%+), you can choose it. You will have high concurrent in read and write operations.

Stack

• If you want to use stack in single thread environment, you can use ArrayDeque and LinkedList, or construct by wrapper ArrayList or array. Selection priority: ArrayDeque > LinkedList
• If you want to use stack with thread-safe. You can use Stack or ConcurrentLikedDeque. The Stack has strictly consistent and poor concurrency. the ConcurrentLinkedDeque has high concurrency and weakly consistent.

Queue

• In single thread environments, the ArrayDeque is the most common queue. If you want a queue has priority, you can use PriorityQueue.
• In multithread environments, There have two type queues: blocking queue and concurrent queue. Blocking queues is commonly using for the producer-consumer scenario. About blocking queue selection, blocking queues for general using you can select LinkedBlockingDeque, ArrayBlockingQueue, and LinkedBlockingQueue. There are many special features blocking queues LinkedTransferQueue, PriorityBlockingQueue, DelayQueue, and SynchronousQueue that for special using scenario. You can use it according to their features.
• Another thread-safe queue type is concurrent queues that have high concurrency. Concurrent queues ConcurrentLinkedQueue and ConcurrentLinkedDeque are very similar, just ConcurrentLinkedDeque can use as both queue and stack.

Set

• In single thread environments, the HashSet is the most common set. Other sets using in single thread environments are LinkedHashSet and TreeSet. The LinkedHashSet is similar to HashSet, but it has additional functionality that keeps elements insert order by linking them up. The TreeSet has lower performance than HashSet, but it can keep elements order with natural order or comparator.
• In multithread environments, We can use thread-safe sets CopyOnWriteArraySet and ConcurrentSkipListSet. The CopyOnWriteArraySet only use when most of operations (90%+) are reads. The ConcurrentSkipListSet use when you want to keep elements Order.

Map

• In single thread environments, the HashMap is the most common map. It’s for general using. Other maps using in single thread environments are LinkedHashMap, EnumMap, IdentityHashMap, WeakHashMap, TreeMap. The LinkedHashMap is similar to HashMap, but it keeps elements insert order. The EnumMap use when all keys of map are from an Enum types. The IdentityHashMap and WeakHashMap are rarely using, just using in special scenario according to their features. The TreeMap is lower performance than HashMap, but it keep elements of map order with natural order or comparators.
• In multithread environments, you can choose Hashtable or concurrent maps. The Hashtable is a thread-safe map by synchronized. It’s strictly consistent and lowly concurrent. If you want to thread-safe and highly concurrent map, you can choose concurrent maps ConcurrentHashMap or ConcurrentSkipListMap. The ConcurrentHashMap is similar to HashMap, but have more concurrent. The ConcurrentSkipListMap have high concurrency and keeps elements order.

String

• String is the most common class for representing a string. It’s final or immutable. If you want to concatenate multiple strings to one, you better use StringBuilder, it doesn’t generate middle strings that saving memory cost.
• The StringBuffer is similar to StringBuilder, the only difference between StringBuffer and StringBuilder that is StringBuffer is thread-safe by synchronized.

Understanding Java articles

• Understanding Java：IO
• Understanding Java：Lambda and Stream
• Understanding Java：Container
• Understanding Java：Concurrency
• Understanding Java：Proxy
• Understanding Java：NIO

References

[1] Java SE 8 API Documentation

[2] Java Source Code - java.util, java.util.concurrent

[3] Why CopyOnWriteArrayList copys when writing?

[4] In what situations is the CopyOnWriteArrayList suitable?

[5] How can CopyOnWriteArrayList be thread-safe?

[6] ReentrantLock Example in Java, Difference between synchronized vs ReentrantLock

[7] Difference between BlockingQueue and TransferQueue

[8] Memory Consistency Errors

–END–

This post will discuss content in Java lambda and stream. It’s based Java SE 8.

Stream

Stream

What is Stream

A sequence of elements supporting sequential and parallel aggregate operations.

Special Stream class (like IntStream) that they provide additional methods for its type of stream (like sum() of IntStream)

To perform a computation, stream operations are composed into a stream pipeline. A stream pipeline consist of source, zero or more intermediate operations, and a terminal operation or forEach(Consume). Streams are lazy; computation on the source data is only performed when the terminal operation is initiated, and source elements are consumed only as needed.

Streams do not provide a means to directly access or manipulate their elements, and are instead concerned with declaratively describing their source and the computational operations which will be performed in aggregate on that source. However, if the provided stream operations do not offer the desired functionality, the BaseStream.iterator() and BaseStream.spliterator() operations can be used to perform a controlled traversal.

A stream should be operated on (invoking an intermediate or terminal stream operation) only once. Since some stream operations may return their receiver rather than a new stream object, it may not be possible to detect reuse in all cases.

Methods of Stream interface

Intermediate Operations

• Filter
  • Stream distinct()
  • Stream filter(Predicate<? super T> predicate)
  • Stream limit(long maxSize)
  • Stream skip(long n)
• Map (or Convert)
  • Stream flatMap(Function mapper)
  • DoubleStream flatMapToDouble(Function mapper)
  • IntStream flatMapToInt(Function mapper)
  • LongStream flatMapToLong(Function mapper)
  • Stream map(Function mapper)
  • DoubleStream mapToDouble(ToDoubleFunction mapper)
  • IntStream mapToInt(ToIntFunction mapper)
  • LongStream mapTOLong(ToLongFunction mapper)
• Operate stream elements
  • Stream peek(Consumer action)
  • Stream sorted()
  • Stream sorted(Comparator comparator)

Terminal Operations

• Predicate

  • boolean allMatch(Predicate predicate)
  • boolean anyMatch(Predicate predicate)
  • boolean noneMatch(Predicate predicate)
  • Optional findAny()
  • Optional findFirst()

• Collecting. Get Converted Object Value

  • R collect(Collector<? super T,A,R> collector)
  • Object[] toArray()
  • A[] toArray(IntFunction generator)

• Reduction. Get Computing Result

  • long count()
  • Optional max(Comparator comparator)
  • Optional min(Comparator comparator)
  • Optional reduce(BinaryOperator accumulator)
  • T reduce(T identity, BinaryOperator accumulator)

• Traverse

  • void forEach(Consumer action)
  • void forEachOrdered(Consumer action)

Static Methods

• static Stream empty()
• static <T> Stream.Builder<T> builder()
• static Stream concat(Stream<? extends T> a, Stream<? extends T> b)
• static Stream generate(Supplier s)
• static Stream iterate(T seed, UnaryOperator f)
• static Stream of (T... values)
• static Stream of (T t)

Methods of BaseStream interface

• void close()
• boolean isParallel()
• Iterator<T> iterator()
• S onClose(Runnable closeHandler)
• S parallel()
• S sequential()
• Spliterator<T> spliterator()
• S unordered()

Methods of IntStream interface

Intermediate Operations

• Stream<Integer> boxed()

Terminal Operations

• OptionalDouble average()
• int sum()

Examples

Map (or Convert)

List<String> letters = Arrays.asList("a", "b", "c");
List<String> collect = letters.stream().map(String::toUpperCase).collect(Collectors.toList());
System.out.println(collect);

int sum = widgets.stream()
    .filter(w -> w.getColor() == RED)
    .mapToInt(w -> w.getWeight())
    .sum();

Testing

Stream<Integer> stream = Stream.of(1, 1, 3);
stream.allMatch(i -> {return i > 1;})

Get Converted Object Value

Stream<Integer> stream = Stream.of(1, 1, 3);
stream.filter(i -> {return i > 1}).collect(Collectors.toList());

Get Computing Result

Stream.of(1, 1, 3).count();

Stream<Integer> stream = Stream.of(1, 2, 3, 2, 1);
int sum = stream.reduce(0, (i, j) -> {
    return i + j;
}));
int product = stream.reduce(1, (i, j) -> {
    return i * j;
});

Traversing Object

Stream<Integer> stream = Stream.of(1, 2, 3);
stream.forEach(i -> System.out.println(i));

Stream<Integer> stream = Stream.of(1, 2, 3, 2, 1);
stream.sorted((o1, o2) -> {
    return o1 - o2;
}).forEachOrdered(System.out::print);

Collector

A mutable reduction operation that accumulates input elements into a mutable result container, optionally transforming the accumulated result into a final representation after all input elements have been processed. Reduction operations can be performed either sequentially or in parallel.

Collectors

Implementations of Collector that implement various useful reduction operations, such as accumulating elements into collections, summarizing elements according to various criteria.

Function

Function

Function is a functional Interface

• R apply(T t)

Example

Function<Integer, Double> half = a -> a/2.0;
System.out.println(half.apply(10));

Predicate

Predicate is a functional Interface

• boolean test(T t)

Example

Predicate<Integer> predicate = i -> {return i > 1;};
System.out.println(predicate.test(2));

public boolean testPredicate(int val, Predicate<Integer> predicate){
    return predicate.test(val);
}
public void test(){
    assertTrue(testPredicate(2, i -> i > 1));
    assertFalse(testPredicate(1, i -> i > 1));
}

Consumer

Consumer is a functional Interface

• void accept(T t)

Example

Consumer consumer = System.out::println;
consumer.accept("hello Consumer");

Supplier

Supplier is a functional Interface

• T get()

Example

Supplier supplier = () -> {return "hello Supplier";};
System.out.println(supplier.get());

public static void testSupplier(Supplier supplier){
    System.out.println(supplier.get());
}
public static void main(String[] args) {
    testSupplier(()->"hello");
}

BinaryOperator

BinaryOperator is a functional interface, and it extends BiFunction<T, U, R>

• R apply(T t, U u)

Lambda Expressions and Method References

what is lambda expression?

Implements anonymous inner class of interface that has a single method and have annotation @FuncationalInterface can using lambda expression.

Example 1

Thread thread = new Thread(new Runnable(){
    void run(){
        System.out.println("annoymous inner class");
    }
});

Thread thread = new Thread(()->{
    System.out.println("lambda expression");
});

Example 2

File dir = new File("{dir_path}");
File[] files = dir.listFiles(new FileFilter(){
    @Override
    public boolean accept(File file){
        return file.getName().endsWith(".log");
    }
});

File dir = new File("{dir_path}");
File[] files = dir.listFiles((file)->{
    return file.getName().endsWith(".log");
});

What is Method Reference?

You use lambda expressions to create anonymous methods. Sometimes, however, a lambda expression does nothing but call an existing method. In those cases, it’s often clearer to refer to the existing method by name. Method references enable you to do this; they are compact, easy-to-read lambda expressions for methods that already have a name.

Kinds of Method References

There are four kinds of method references:

Examples of Usage

public class People {

    private String name;

    public People(String name) {
        this.name = name;
    }

    public String getName() {
        return this.name;
    }

    public boolean compareName(String name) {
        return this.name == name;
    }

    public static String convertToUpperCase(String s) {
        return s.toUpperCase();
    }

    public static void main(String[] args) {
        // Reference to an instance method of an arbitrary object of a particular type
        Function<People, String> getName = People::getName;
        System.out.println(getName.apply(new People("Tom"))); // Tom

        // Reference to an instance method of a particular object
        People people = new People("Jack");
        Function<String, Boolean> compareName = people::compareName;
        System.out.println(compareName.apply("Jack")); // true

        // Reference to a static method
        Function<String, String> convertToUpperCase = People::convertToUpperCase;
        System.out.println(convertToUpperCase.apply("abc")); // ABC

        // Reference to a constructor
        Function<String, People> getInstance = People::new;
        System.out.println(getInstance.apply("John").getName()); // John
    }
}

Java Util

Optional

What is Optional?

Convert a object to Optional object, if the value is not null, isPresent() will return true and get() return the value; if the value is null, isPresent() will return false and get() return empty Optional object.

In short, you can either get value or empty optional from an optional object, and you never get null.

Optional Methods

Static Methods

• static Optional empty() // get empty Optional
• static Optional of(T value) // if value is null, it will throw NullPointerException
• static Optional ofNullable(T value) // if value is null, return empty Optional

Intermediate Operations

• Optional filter(Predicate predicate)
• Optional flatMap(Function mapper)
• Optional map(Function mapper)

Terminal Operations

• Test
  • void ifPresent(Consumer consumer)
  • boolean ifPresent()
• Get
  • T get()
  • T orElse(T other)
  • T orElseGet(Supplier other)
  • T orElseThrow(Supplier exceptionSupplier)

Example

int getNotNullValueOrDefault(Integer obj, int defaultValue){
    Optional<Integer> optional = Optional.ofNullable(obj);
    return optional.orElse(defaultValue);
}

Comparator

Comparator is a functional interface

• int compare(T o1, T o2)

Spliterator

Spliterator is for traversing and partitioning elements of a source. The source of elements covered by a Spliterator could be, for example, an array, a Collection, an IO channel, or a generator function.

When used in a parallel programming solution, bear in mind that the individual Spliterator is not thread safe; instead the parallel processing implementation must insure that each individual Spliterator is handled by one thread at a time. A thread calling the trySplit() of a Spliterator, may hand over the returned new Spliterator to a separate thread for processing. This follows the idea of decomposition of a larger task into smaller sub-tasks that can be processed in parallel individually of each other.

Note Spliterator likes Stream every operate only once. Can’t traverse the same spliterator more than once.

Methods of Spliterator

• Characteristics
  • int characteristics()
  • default boolean hasCharacteristics(int characteristics)
• Traversal
  • default void forEachRemaining(Consumer action)
  • boolean tryAdvance(Consumer action)
• Others
  • long estimateSize()
  • default long getExactSizeIfKnow()
  • default Comparator getComparator()
  • Spliterator trySplit()

Example

characteristics

List<Integer> list = new ArrayList<>(Arrays.asList(1, 2, 3));
Spliterator<Integer> spliterator = list.spliterator();
System.out.println(spliterator.hasCharacteristics(Spliterator.SIZED)); // true
int expected = Spliterator.ORDERED | Spliterator.SIZED | Spliterator.SUBSIZED;
System.out.println(spliterator.characteristics() == expected); // true

traversal

spliterator.forEachRemaining(System.out::println);

while (spliterator.tryAdvance(System.out::println));

split

List<Integer> list = new ArrayList<>(Arrays.asList(1, 2, 3));
Spliterator<Integer> spliterator = list.spliterator();
Spliterator<Integer> newPartition = spliterator.trySplit();
spliterator.forEachRemaining(System.out::println);
System.out.println("======");
newPartition.forEachRemaining(System.out::println);

Summary

Stream is operations on collections, and is similar to multiple times “for loop”, but it’s more concise than multiple times “for loop” and it provides many convenient operations implementation. Additionally, it can sequential or parallel operating container elements.

Functional Interface represents the class that can be use lambda expression.

Lambda Expression is similar to implementation of anonymous inner class, but it’s more concise than anonymous inner class.

Method Reference is similar to lambda expression, but it’s more concise than Lambda Expression. It is a replacement of method implementation of an anonymous inner class when they have the same input and output of the method. It’s more convenient and concise than Lambda Expression.

Understanding Java articles

• Understanding Java：IO
• Understanding Java：Lambda and Stream
• Understanding Java：Container
• Understanding Java：Concurrency
• Understanding Java：Proxy
• Understanding Java：NIO

References

[1] Java API docs

[2] Lambda Expressions - The Java Tutorials

[3] Method References - The Java Tutorials

This post will discuss content in Java IO streams. It is based Java SE 8.

Input and Output Stream Hierarchy

Input Stream

Input Stream Methods

• int available(): return available to read the remaining byte size.
• void close(): close the input stream.
• void mark(int readlimit): marks the current position in this input stream.
  • readlimit: the number of bytes that can be read from the stream after setting the mark before the mark becomes invalid. Note the parameter for some subclass of InputStream has no meaning.
• boolean markSupported(): return if this input stream supports the mark and reset methods.
• int read(): reads the next byte of data.
• int read(byte[] b): reads some bytes from input stream and stores them into buffer array b.
• int read(byte[] b, int off, int len): reads up to len bytes of data from input stream into array of bytes.
• void reset(): repositions this stream to the position at mark (default 0)
• long skip(long n): skips bytes of data.

What does Input Stream actually do?

InputStream only reads data from front to back. It can skip some data to read. It can back to specified position to read again.

InputStream can read bytes one by one or read some bytes into byte array once.

If there is no available data in InputStream, the calling one of the read methods will return -1.

int read(byte b[]) => int read(byte b[], int offset, int len), abstract int read()

read() methods can implement directly (e.g ByteArrayInputStream) or implement by calling the native method (e.g FileInputStream).

InputStream Types

• ByteArrayInputStream: Byte array as data
  • ByteArrayInputStream(byte[] b)
• FileInputStream: File as data
  • FileInputStream(File file)
• ObjectInputStream: Deserializes primitive data and objects previously written using an ObjectOutputStream.
  • ObjectInputStream(InputStream in)
  • readObject()
• PipedInputStream: A piped input stream should be connected to a piped output stream; the piped input stream then provides whatever data bytes are written to the piped output stream.
  • PipedInputStream()
  • connect(PipedOutputStream src)
• StringBufferStream(Deprecated): String as data.
  • StringBufferInputStream(String s)
• SequenceInputStream: Logical concatenation of other input streams.
  • SequenceInputStream(InputStream s1, InputStream s2)
• FilterInputStream: It contains some other input stream as data, and transforming data or providing additional functionality.
  • BufferedInputStream: read data from the internal buffer array rather than read data from input stream every time.
    • BufferedInputStream(InputStream in)
  • PushbackInputStream: unread one byte.
    • PushbackInputStream(InputStream in)
    • unread(int byteValue)
  • DataInputStream: read primitive Java data types.
    • DataInputStream(InputStream in)
    • readInt()
  • LineNumberInputStream(Deprecated): provides the added functionality of keeping track of the current line number.
    • LineNumberInputStream(InputStream in)
    • getLineNumber()

OutputStream

OutputStream Methods

• void close(): close output stream and release resources.
• void flush(): Flushes output stream and forces buffered bytes to be written out.
• void write(byte[] b): Writes byte array.
• void write(byte[] b, int offset, int len): Writes len bytes from the byte array starting at offset.
• void write(int b): Writes the specified byte.

What does Output Stream actually do?

Write one byte or some bytes data to output stream.

If output stream doesn’t call the flush or close method, some data written in buffer array may don’t write to the stream.

Some output streams have the buffer array, others not.

OutputStream Types

• ByteArrayOutputStream: Writes data to a byte array. retrieve data using toByteArray() and toString().
  • toByteArray()
  • toString()
• FileOutputStream: Writes data to a File or FileDescriptor.
  • FileOutputStream(File file)
• ObjectOutputStream: Writes primitive data types and java objects to an OuputStream.
  • ObjectOutputStream(OutputStream out)
  • writeObject(Object obj)
  • writeInt(int val)
• PipedOutputStream: Sending data to pipe.
• FilterOutputStream: Sit on top of an already existing output stream. It transforming the data or providing additional functionality.
  • BufferedOutputStream: Writes data to buffer array. Avoid write data to stream every time.
    • BufferedOutputStream(OutputStream out)
  • DataOutputStream: Write primitive Java data types to output stream.
    • writeChars(String s)
    • writeInt(int val)
  • PrintStream: Print various values to output stream.
    • println(String s)
    • write(int b)

Notes

if not implements Serializable interface, calling writeObject() will throw java.io.NotSerializableException

Reader and Writer Hierarchy

Reader

Reader Methods

• abstract void close()
• void mark(int readAheadLimit)
• boolean markSupported()
• int read()
• int read(char[] cbuf)
• abstract int read(char[] cbuf, int off, int len)
• int read(CharBuffer target)
  • java.nio.CharBuffer target is using for read characters.
• boolean ready(): return whether this stream is ready to be read. It’s same with InputStream.available()
• void reset()
• long skip(long n)

What does Reader actually do?

Read one character to return or read some characters into char array from somewhere, e.g char array, file, pipe.

Reader Types

Description like InputStream

• BufferedReader
  • BufferedReader(Reader in)
  • readLine()
• LineNumberReader
  • LineNumberReader(Reader in)
  • readLine()
  • getLineNumber()
• CharArrayReader
  • CharArrayReader(char c[])
• Abstract FilterReader
• PushbackReader
  • PushbackReader(Reader in)
  • unread(int val)
• InputStreamReader
  • InputStreamReader(InputStream in)
• FileReader
  • FileReader(File file)
• PipedReader
  • connect(PipedWriter src)
• StringReader
  • StringReader(String s)

Writer

Writer Methods

• Writer append(char c)
• Writer append(CharSequence csq)
  • Same with write(String str), but it can chained call, and String parameter can be null.
• Writer append(CharSequence csq, int start, int end)
• abstract void close()
• abstract void flush()
• void write(char[] cbuf)
• abstract void write(char cbuf, int offset, int len)
• void write(int c)
• void write(String str)
• void write(String str, int offset, int len)

What does Writer actually do?

Writes one character or write some characters to somewhere, e.g char array, file, pipe.

Writer Types

• BufferedWriter
  • BufferedWriter(Writer out)
  • void newLine()
• OutputStreamWriter
  • OutputStreamWriter(OutputStream out, Charset cs)
• FileWriter
  • FileWriter(File file)
• StringWriter
  • StringBuffer getBuffer()
• CharArrayWriter
  • String toString()
  • char[] toCharArray()
• abstract FilterWriter
• PipedWriter
• PrintWriter
  • PrintWriter(OutputStream out)
  • PrintWriter(Writer out)
  • void println(String s)

Standard IO Streams

java.lang.System

• PrintStream out
• InputStream in
• PrintStream err

Interfaces

java.lang.AutoClosable: Auto close resources.

• void close(): This method is invoked automatically on objects managed by the try-with-resources statement.

Closeable: resources can be closed.

• void close()

DataInput: provide reading data in any of the Java primitive types.

• boolean readBoolean()
• int readInt()
• …

DataOutput: provide writing data in any of the Java primitive types.

• void writeBoolean(int b)
• void writeInt(int val)
• …

Externalizable: extends Serializable interface. It using for object serialization.

• void readExternal(ObjectInput in)
• void writeExternal(ObjectOutput out)

FileFilter: It is a functional interface and can be used for a lambda expression or method reference. Filter files.

• boolean accpet(File pathname)

FileNameFilter: It is a functional interface and can be used for a lambda expression or method reference. Filter file names.

• boolean accept(File dir, String name)

Flushable: data can be flushed.

• void flush(): Flushes buffered output to the underlying stream.

ObjectInput: extends DataInput interface. Allows reading object.

• Object readObject()

ObjectInputValidation: Callback interface to allow validation of objects within a graph. It has no implemented subclasses.

ObjectOutput: extends DataOutput interface. Allows writing object.

• void writeObject(Object obj)

ObjectStreamConstants: Constants written into the Object Serialization Stream.

Serializable: Allows class serializability. It has no methods.

File

File

• Fields
  • static String pathSeparator: system dependent path separator, e.g ;.
  • static String separator: system dependent name separator, e.g /.
• Constructors
  • File(File parent, String child), File(String pathname), File(String parent, Sring child), File(URI uri)
• File Permission
  • boolean canExecute(), boolean canRead(), boolean canWrite()
  • boolean setExecutable(boolean executable), boolean setWritable(boolean writable)
• File Handling
  • static File createNewFile() , static File createTempFile(String prefix, String suffix, File directory), boolean mkdir(), boolean mkdirs(), renameTo(File dest)
  • boolea delete(), boolean deleteOnExit()
  • int compareTo(File pathname): compares two abstract pathnames lexicographically.
  • String[] list(), String[] list(FilenameFilter filter), File[] listRoots()
  • Path toPath(), URI toURI(), URL toURL()
• File Information
  • boolean exists()
  • isAbsolute(), isDirectory(), isFile(), isHidden()
  • String getAbsolutePath(), String getCanonicalPath(), String getName(), File getParentFile(), String getPath()
  • long getFreeSpace(), long getTotalSpace(), long getUsableSpace(), long lastModified(), long length()

FileDescriptor

Whenever a file is opened, the operating system creates an entry to represent this file and stores its information. Each entry is represented by an integer value and this entry is termed as file descriptor. [2]

Basically, Java class FileDescriptor provides a handle to the underlying machine-specific structure representing an open file, an open socket, or another source or sink of bytes. [2]

The applications should not create FileDescriptor objects, they are mainly used in creation of FileInputStream or FileOutputStream objects to contain it. [2]

File Paths

Get file, absolute filepath and inputStream in classpath, e.g src/resources/test.txt

You can get InputStream by using ClassLoader.getResourceAsStream() method.

InputStream in = getClass().getClassLoader().getResourceAsStream("test.txt");

You can get file object and file path by using ClassLoader get URL, then get them.

URL url = getClass().getClassLoader().getResource("test.txt");
File file = new File(url.getFile());
String filepath = file.getAbsolutePath();

Difference between getResource() methods:

getClass().getClassLoader().getResource("test.txt") //relative path
getClass().getResource("/test.txt")); //note the slash at the beginning

Get file, absolute file path and inputStream in JAR, e.g src/resources/test.txt

This is deliberate. The contents of the “file” may not be available as a file. Remember you are dealing with classes and resources that may be part of a JAR file or other kind of resource. The classloader does not have to provide a file handle to the resource, for example the jar file may not have been expanded into individual files in the file system. [3]

Anything you can do by getting a java.io.File could be done by copying the stream out into a temporary file and doing the same, if a java.io.File is absolutely necessary. [3]

Result: You can get input stream and File by ClassLoader, but you can’t get right usable absolute file path. If you want get a usable absolute path in JAR, you can copy resource file stream to create a new temporary file, then get absolute file path of the temporary file.

File file = null;
String resource = "/test.txt";
URL res = getClass().getResource(resource);
if (res.getProtocol().equals("jar")) {
    try {
        InputStream input = getClass().getResourceAsStream(resource);
        file = File.createTempFile("tempfile", ".tmp");
        OutputStream out = new FileOutputStream(file);
        int read;
        byte[] bytes = new byte[1024];

        while ((read = input.read(bytes)) != -1) {
            out.write(bytes, 0, read);
        }
        out.close();
        file.deleteOnExit();
    } catch (IOException ex) {
        Exceptions.printStackTrace(ex);
    }
} else {
    //this will probably work in your IDE, but not from a JAR
    file = new File(res.getFile());
}

System.out.println(file.getAbsolutePath());

Summary

IO Streams Functionality

InputStream or Reader it read data from somewhere, e.g byte or char array, file, pipe. It can read single-unit data from the stream to return or read multiple units data from the stream to store into an array every time. read() methods can implement by java code or implements by calling native method. Filter streams use the Decorator Pattern to add additional functionality (e.g unread) that implements by internal buffer array to other stream objects.

OutputStream or Writer it write data to somewhere, e.g byte or char array, file, pipe.

IO Stream Types

• Byte or Char Array
• File
• Object
• Piped
• Sequence:
• Filter
  • Buffered: Operating with buffer array.
  • DataInput, DataOutput: Operating with primitive Java types
  • Pushback: Add unread functionality.
  • Print:
  • LineNumber

Others

class java.nio.CharBuffer using in method int read(CharBuffer target)

interface java.lang.CharSequence using in method Writer append(CharSequence csq)

Questions

InputStream vs Reader? What the real difference between them? Who has more efficient?

Stream is operating bytes, and reader or writer is operating characters. Writing streams of raw bytes such as image data using OutputStream. Writing streams of characters using Writer.

CharSequence vs String

A CharSequence is a readable sequence of char values. You can call chars() method get the InputStream.

Understanding Java articles

• Understanding Java：IO
• Understanding Java：Lambda and Stream
• Understanding Java：Container
• Understanding Java：Concurrency
• Understanding Java：Proxy
• Understanding Java：NIO

References

[1] Java SE 8 API Document

[2] Java File Descriptor Example

[3] How to get a path to a resource in a Java JAR file - Stack Overflow

Content

• Introduction to Servlet
• The Servlet Interface
• The Request
• Servlet Context
• The Response
• Filtering
• Sessions
• Dispatching Requests
• Web Applications
• Application Lifecycle Events
• Mapping Requests to Servlets
• Security

Introduction to Servlet

What is a Servlet

Servlet 是基于 Java 的 Web component，它是被 servlet container 管理的，它可以用来生成动态的内容。Servlet 是平台独立的 java class，可以运行在支持 servlet container 的 Web Server 中。Servlet 通过 Servlet 容器实现的 request/response 范例与 Web 客户端进行交互。

What is a Servlet Container

Servlet container （有时也叫做 servlet engine）它是 Web server 或 application server 的一部分，它提供了发送 request 和 response 的网络服务，解析基于 MIME 的 requests，以及格式化基于 MIME 的 responses。Servlet Container 还管理 Servlets 的整个 lifecycle。

Servlet container 可以内置到 Web server 中，也可以通过 Web server 的扩展 API 作为附加组件安装到 Web server 中。Servlet container 也可以内置或安装在 application servers。

所有 servlet containers 必须支持 HTTP 作为请求和响应的协议。其它基于 request/response 的协议也可能支持如 HTTPS。

Why do we need Servlet

实现基于 HTTP 协议的 Java Web 应用程序，我们需要使用 servlet 技术来生成动态的响应内容。

Why do we need Servlet Container

Servlet 只是一个 Java 类，它接收 request 对象和响应 response 对象。然而，一个应用中有很多 servlets，我们需要一个容器来管理这些 servlets 的创建和销毁，以及去解析和生成网络协议的报文。

How they work

• Client 发出一个 HTTP 请求访问 Web server。
• Web server 接收到请求，将请求交给 servlet container。servlet container 可以在与主机 Web server 相同的进程中运行，可以在同一主机上的不同进程中运行，也可以在与其处理请求的 Web server 不同的主机上运行。
• Servlet container 根据配置决定调用哪个 servlet，并将表示 request 和 response 的对象传递给 servlet。
• Servlet 利用 request 对象处理逻辑，生成响应的数据。
• 一旦 servlet 处理完了请求，servlet container 确保 response 正确地 flushed，并且将控制权返回给 Web server。

Servlet History

Servlet versions history

Servlet versions and apache tomcat versions

Servlet Example: HelloWorld

1.Generating maven project

$ mvn archetype:generate -DgroupId=com.taogen.example -DartifactId=servlet-helloworld -DarchetypeArtifactId=maven-archetype-webapp -DinteractiveMode=false

2.Add servlet-api dependencies in pom.xml

<dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>2.5</version>
    <scope>provided</scope>
</dependency>

3.Add HelloWroldServlet.java

package com.taogen.example;

import java.io.*;
import java.util.Date;
import javax.servlet.*;
import javax.servlet.http.*;

// Extend HttpServlet class
public class HelloWorldServlet extends HttpServlet {
 
   private String message;

   public void init() throws ServletException {
      // Do required initialization
      message = "Hello World! ";
   }

   public void doGet(HttpServletRequest request, HttpServletResponse response)
      throws ServletException, IOException {
      
      // Set response content type
      response.setContentType("text/html");

      // Actual logic goes here.
      PrintWriter out = response.getWriter();
      out.println("<h3>" + this.message + "</h3>");
   }

   public void destroy() {
      // do nothing.
   }
}

4.Configuring servlet in web.xml

<servlet>
    <servlet-name>HelloWorld</servlet-name>
    <servlet-class>com.taogen.example.HelloWorldServlet</servlet-class>
</servlet>
<servlet-mapping>
    <servlet-name>HelloWorld</servlet-name>
    <url-pattern>/HelloWorld</url-pattern>
</servlet-mapping>

5.Running project

Package project by mvn package, move war file to Apache Tomcat /webapps directory, starting Apache Tomcat server.

Visiting the URL http://localhost:8080/{your-servlet-context}/HelloWrold

The Servlet Interface

Servlet interface 是 Java Servlet API 的核心抽象。所有的 servlets 直接或间接的实现了这个接口。Java Servlet API 中有两个实现了 Servlet interface 的类 GenericServlet 和 HttpServlet。一般开发人员通过 extends HttpServlet 来实现它们 Servlets。

Request Handing Methods

基础的 Servlet interface 定义了 service 方法来处理客户端请求。对于 servlet container 路由到 servlet 实例的每个请求，都会调用此方法。

处理 Web 应用程序的并发请求，通常需要 Web Developer 设计可以处理多个线程执行 service 方法的servlets。

通常 Web container 通过在不同的线程上并发执行 service 方法来处理对同一 servlet 的并发请求。

HTTP Specific Request Handling Methods

Servlet 接口的抽象子类 HttpServlet 添加了额外的方法来帮助处理基于 HTTP 的请求。这些方法是：

• doGet
• doPost
• doPut
• doDelete
• doHead
• doOptions
• doTrace

Number of Instances

在非分布式环境，servlet container 中每个 servlet 仅能有一个实例。如果 servlet 实现 SingleThreadModel 接口，servlet container 可能会实例化多个 servlet 实例。

SingleThreadModel 保证仅有一个线程执行 servlet 实例的 service 方法，它避免并发地访问一个 servlet 实例的 service 方法，然而，我们可以通过其它方法达到这个目的，SingleThreadModel 是不推荐使用的。

Servlet Life Cycle

Servlet 是通过定义明确的生命周期进行管理的，该生命周期定义了如何加载和实例化，如何初始化，如何处理来时客户端的请求，和如何退出服务。API 中的生命周期由 javax.servlet.Servlet 接口的 init，service 和 destroy 方法表示，所有的 servlet 必须直接或者通过 GenericServlet 或 HttpServlet 抽象类间接地实现这个接口。

Loading and Instantiation

Servlet container 负责加载和实例化 servlet。加载和实例化可以在 container 启动的时候，或者延迟到 container 需要 servlet 来处理请求。Servlet 默认是懒加载。

Initialization

在 servlet 对象实例化后，在它能处理客户端请求之前，container 必须初始化 servlet。初始化方便 servlet 可以读取持久性配置数据，初始化昂贵的资源以及执行其它一次性的活动。container 通过实现 ServletConfig 接口唯一（每个 Servlet 声明）对象调用 Servlet 接口的 init 方法开初始化 servlet。配置对象允许 servlet 从 Web 应用配置信息中访问 name-value 初始化参数。

Request Handling

在一个 servlet 正确初始化后，servlet container 可能使用它来处理客户端的请求。请求通过 ServletRequest 对象来表示，servlet 通过调用 ServletResponse 对象提供的方法来响应请求。这两个对象作为参数传递给Servlet 接口的 service 方法。

对于 HTTP 请求，container 提供的对象类型是 HttpServletRquest 和 HttpServletResponse。

Multithreading Issues

Servlet container 可能通过 servlet 的 service 方法发送并发请求。为了处理这些请求，Servlet 开发者必须为 service 方法中的多线程并发处理做好充分的准备。

Servlet 的 service 方法不建议使用 synchronized 关键字，因为那将使得 container 不能使用 instance pool，而是顺序执行请求，这会严重影响 servlet 的性能。

Exception During Request Handling

Servlet 在请求 service 时，可能 throw ServletException 或 UnavailableException，其中 ServletException 表示在处理请求过程中由错误发生，并且 container 应该采取适当措施来清理请求。UnavailableException 表示这个 servlet 临时或永久地不能处理请求，container 必须从服务中移除这个 servlet，调用它的 destroy 方法，并且释放 servlet 实例。

Thread Safety

request 和 response 对象的实现没有保证线程安全，这意味着它们应该在请求处理线程范围内使用。request 和 response 对象的引用不应该执行在其它线程。

End of Service

servlet container 不需要在任何特定时间内都保持 servlet 的加载。Servlet 实例在 servlet 容器中的生命周期可能是几天，几个月或者几年。

当 servlet container 决定把一个 servlet 从 service 中移除，它调用 Servlet 接口的 destroy 方法去允许这个 servlet 去释放所有它使用的和保存的任何持久状态的资源。

在 servlet container 调用destroy 方法，它必须允许任何正在执行这个 servlet 的 service 方法的线程执行完毕，或者执行超时。

一旦一个 servlet 实例的 destroy 方法被调用了，这个实例将不再接收请求。如果 container 需要这个servlet，它必须重新创建一个新的实例。在 destroy 方法完成后，servlet container 必须释放 servlet 实例，让它能够进行垃圾收集。

The Request

Request object 封装了来自客户端请求的所有信息。对于 HTTP 协议，这个信息是从客户端传递到服务器的 HTTP 请求的 headers 和 message body。

HTTP Protocol Parameters

Parameters 是由一组 name-value pair 存储的。ServletRequest 接口中获取参数的方法有：

• getParameter
• getParameterNames
• getParameterValues
• getParameterMap

来自 query string 和 post body 的数据是聚合在 request parameter set 中的。query string 数据表示在post body 数据之前。如：一个请求的 query string 是 http://xxx?a=hello，它的 post body 是 a=goodbye&a=world，参数集的结果将是 a=(hello, goodbye, world)。

Post 表单数据转换为 parameter set 的条件：

1. 它是一个 HTTP or HTTPS 请求。
2. HTTP method 是 POST。
3. content type 是 application/x-www-form-urlencoded
4. Servlet 已对请求对象上的任何 getParameter 方法族进行了初始调用。

如果上面的条件没有全部满足，post 请求的 form data 不会包含在 parameter set中，但 post data 依然可以通过 request object 的 input stream 中获取。如果所有条件都满足，post form data 将不再能从 request object 的 input stream 中读取。parameter 只能表现为 form data 和 input stream 两种方式之一。

Attributes

Attributes 是关联一个请求的对象。Container 可以设置 attributes 以表示无法通过 API 表示的信息，或者可以由 servlet 设置 attributes 以将信息传递给另一个 servlet （通过 RequestDispatcher）。ServletRequest 接口操作 attributes 的方法有：

• getAttribute
• getAttributeNames
• setAttribute

一个 attribute value 只能与一个 attribute name 关联。

以 “java.” 和 “javax.” 为前缀的 attributes 是 servlet specification 定义的。

Headers

HttpServletRequest 接口获取 header 的方法有：

• getHeader
• getHeaders
• getHeaderNames

可能存在多个 headers 是相同的名称，如果有多个 header 是相同的名称，getHeader 方法返回第一个 header，getHeaders 方法返回所有 headers 的 Enumeration 对象。

Headers 可能 string 表示的 int 和 Date 数据，HttpServletRequest 接口提供了直接获取 int 和 Date 类型的 header 的方法：

• getIntHeader
• getDateHeader

getIntHeader 方法可能会 throw NumberFormatException，getDateHeader 方法可能 throw IllegalArgumentException。

Request Path Elements

request URI = Context Path + Servlet Path + Path Info + Query String

e.g. http://myserver.com/myproject/myservlet/remove?id=1

Configuration

• <servlet-mapping>myservlet/*</servlet-mapping>

Result

• Context Path: /myproject
• Servlet Path: /myservlet
• Path Info: /remove
• Query String: ?id=1

Path Translation Methods

Servlet API 中允许开发者获取 Java Web 应用的文件在文件系统中的绝对文件路径。这些方法是：

• ServletContext.getRealPath
• HttpServletRequest.getPathTranslated

Cookies

HttpServletRquest 接口提供了 getCookies 方法去获取请求中的一组 cookies。这些 cookies 是从客户端每次发送到服务端的请求中的数据。

服务端可以添加和删除 Cookie， 以及设置 cookie 的属性，如有效期等。

Internationalization

客户端可以选择向 Web server 指示它们希望以哪种语言给出响应。客户端可以使用 header 中的 Accept-Language 属性来传达这个信息。ServletRequest 接口提供了获取客户端的偏好语言的方法：

• getLocale
• getLocales

getLocale 方法返回客户端最希望的语言的 locale 对象。getLocales 方法返回 locale 对象的 Enumeration，以降序的方式表示客户端所有偏好的语言。

如果客户端没有指明偏好的语言，那么 getLocale 将返回 servlet container 默认的 locale，getLocales 将返回只包含 默认 locale 的 enumeration。

Request data encoding

如果客户端没有通过 Content-Type header 指明 HTTP request 的字符编码，HttpServletRequest.getCharacterEncoding 方法将返回 null。

Container 读取 request 的数据的默认编码为 ISO-8859-1。开发者可以通过 setCharacterEncoding 方法来设置 request 的字符编码。设置 request 字符编码一定要在读取 request 数据之前，一旦数据被读取了，字符编码的设置将不会生效。

Lifetime of the Request Object

每一个 request 对象仅仅在 servlet 的 service 方法或者 filter 的 doFilter 方法范围内有效。Container 为了减少创建 request 对象的性能花费，通常会循环利用 request 对象。开发者必须注意，在非有效范围之外维持 request 对象的引用是不推荐的，它可能导致意外的结果。

Servlet Context

SevletContext interface 定义了一组方法让 servlet 与 它的 servlet container 进行交流。例如，获取一个文件的 MIME type，dispatch request，写日志，以及设置和存储所有 servlet 可以访问的属性等。

Scope of ServletContext Interface

每一个 Java 虚拟机的每一个 Web 应用程序只有一个 ServletContext 对象。

Initialization Parameters

ServletContext 接口允许 servlet 访问在 deployment descriptor 定义的 context 初始化参数的方法：

• getInitParamenter
• getInitParamenterNames

应用程序开发人员使用初始化参数来表示设置信息。如网站管理员的电子邮件地址，或者系统的关键配置数据。

Context Attributes

Servlet 可以通过 name 把对象属性绑定到 context 中。任何绑定到 context 中的 attribute Web 应用程序中任何其他 servlet 都可以使用。ServletContext 接口操作 attributes 的方法：

• setAttribute
• getAttribute
• getAttributeNames
• removeAttribute

Resources

ServletContext interface 提供了直接访问静态类型文档，如 HTML，GIF，和JPEG 等文件的方法：

• getResource
• getResourceAsStream

静态文件的 path 是以‘/’开始的 context 根目录的相对路径。上面的方法不能获取动态文件如 getResource(“/index.jsp”)

Temporary Working Directories

Servlet containers 必须为 每一个 servlet context 提供一个私有的临时目录，并且可以通过 javax.servlet.context.tempdir context attribute 来访问这个目录。

The Response

Response object 封装了 server 返回给 client 的所有信息。在 HTTP 协议中，这个信息通过 HTTP headers 或 message body 从 server 传输到 client。

Buffering

出于效率目的，允许（但不是必需）servlet container 来缓冲输出到客户端的数据。一般服务器是默认缓冲的，允许 servlet 去指定 buffering 的参数。

ServletResponse interface 允许 servlet 访问和设置 buffering 信息的方法：

• getBufferSize
• setBufferSize
• isCommitted
• reset
• resetBuffer
• flushBuffer

ServletResponse interface 提供这些方法去执行缓冲操作，无论 servlet 使用 ServletOutput 还是 Writer。

Headers

Servlet 可以通过 HttpServletResponse interface 的方法去设置 HTTP repsose 的 headers：

• setHeader
• addHeader

HttpServletResponse interface 也提供了添加具体的数据类型的 headers 的方法：

• setIntHeader
• setDateHeader
• addIntHeader
• addDateHeader

Convenience Methods

HttpServletReponse interface 中的 convenience 方法有：

• sendRedirect
• sendError

sendRedirect 是完整的 request URI，它包含 context path 即 /contextPath/servletPath/pathInfo，而 RequsetDispather 的 forward 和 include 是相对于 context path 路径的相对目录，即 /servletPath/pathInfo

Internationalization

Servlet 可以设置 response 的 locale 和 character encoding。

Servlet 使用 ServletResponse 的 setLocale 方法可以设置 response 的 locale。如果 response 已经 committed 则 setLocale 方法是无作用的。如果 servlet 在 response committed 之前没有设置 locale，container 的默认的 locale 用于确定 response 的 locale。

Servlet 可以使用 locale encoding mapping 来设置使用特定 locale 时也使用对应的 character encoding。

<locale-encoding-mapping-list>
	<locale-encoding-mapping>
		<locale>zh-CN</locale>
		<encoding>UTF-8</encoding>
	</locale-encoding-mapping>
</locale-encoding-mapping-list>

ServletResponse 提供了设置 character encoding 的方法：

• setCharacterEncoding
• setContentType

和 setLocale 方法一样 set character encoding 在 response committed 之后是无作用的。如果 servlet 在 ServletResponse 的 getWriter 方法调用之前或 response committed 之后，没有设置 character encoding，则默认使用 ISO-8859-1 编码。

Closure of Response Object

当 response 关闭时，container 必须立刻 flush 在 response buffer 中所有剩余的内容，返回给 client。

关闭 request 和 response 对象的 events：

• servlet 的 service 方法结束时。
• 在 response 的 setContentLength 方法指定的内容量大于零，，并已写入 response 中。
• 调用了 sendError 方法时。
• 调用了 sendRedirect 方法时。

Lifetime of Response Object

每个 response 对象仅仅在 servlet 的 service 方法，或者在 filter 的 doFilter 方法范围内有效。Container 通常会循环利用 response 对象来减少创建 response 对象的性能消耗。开发者必须注意在 response 对象的有效范围之外维护 response 对象的参考可能会导致意外的行为。

Filtering

Filter 是 Java Servlet 组件，它允许在接受和响应请求的过程中访问和修改请求的 header 和 payload。

What is a Filter

Filter 是一段重用的代码，是一个 java class，它可以转换 HTTP request, response, 和 header 的内容。Filter 通常不会像 servlet 那样创建 response 或 响应请求，而是修改和调整对资源的请求，并修改或调整对资源的响应。

Filter 可以作用在动态或静态的内容上。动态和静态的内容一般指的是 Web resources。

开发者使用 filter 功能的类型有：

• 在请求调用之前访问资源。
• 在请求调用之前处理请求。
• 通过用自定义的 request 对象 wrapping request 来修改 request 的 headers 和 data。
• 通过用自定义的 response 对象来修改 response 的 headers 和 data。
• 调用资源后对其进行拦截。
• 对一个或一组 servlet 按顺序执行多个 filter 的操作。

使用 Filter components 常见的例子：

• Authentication filters
• Logging and auditing filters
• Image conversion filters
• Data compression filters
• Encryption filters
• Tokenizing filters
• Filters that trigger resource access events
• XSL/T filters that transform XML content
• MIME-type chain filters
• Caching filters

Main Concepts

开发者通过 implement javax.servlet.Filter interface 创建 filter，并且提供一个无参的 constructor。Filter 在 deployment descriptor 中使用 <filter> 进行声明。一个或一组 filter 可以通过在 deployment descriptor 中定义 <filter-mapping> 元素配置它的调用。这个配置通过 servlet 的 logic name 或者 resources URL 来实现。

Filter Lifecycle

在 Web 应用程序部署之后，container 接收到请求之前，container 必须找到应用于 Web 资源的 filter 列表。container 必须确保每个 filter 实例化，并且调用 init(FilterConfig config) 方法进行初始化。

每一个 filter 只有一个实例。当 container 接收请求，它传递 ServletRequest 和 ServletResponse 参数调用第一个 filter 实例的 doFilter 方法，并且 FilterChain 对象将被用于传递请求到下一个 filter。

Wrapping Requests and Responses

Filter 概念的核心是 wrapping 请求和响应，以便它可以重写行为来执行过滤任务。开发者不仅可以重写 request 和 response 对象存在的方法，还可以提供新的 API 去满足特定的过滤任务。

当一个 filter doFilter 方法被调用时，container 必须保证传递给下一个 filter 或者目标 web resource 的 request 和 response 对象与传递给当前 doFilter 方法的 request 和 response 对象是相同的。 另外，wrapper object 相同的要求也应用于从 servlet 或 filter 到 RequestDispatcher.forward or include 的调用。

Filter Environment

Filter 的初始化参数在 deployment descriptor 中<filter> 内的 <init-params> 元素中定义。Filter 通过 FilterConfig 的 getInitParameter 方法访问参数。另外，FilterConfig 为了加载资源，logging，存储状态到 ServletContext attribute 中等功能，它可以访问 ServletContext 对象。

Configuration of Filters in a Web Application

For example:

<filter>
    <filter-name>My Filter</filter-name>
    <filter-class>com.example.MyFilter</filter-class>
</filter>
<filter-mapping>
	<filter-name>My Filter</filter-name>
    <servlet-name>MyServlet1</servlet-name>
    <url-pattern>/foo/*</url-pattern>
</filter-mapping>

Filters and the RequestDispatcher

Java Servlet 2.4 之后可以配置 filter 在调用 request dispatcher forward() and include() 方法时过滤。使用 <dispatcher> 元素在指出过滤请求的条件，它的值有：

• REQUEST：表示请求直接来自 client 时过滤。
• FORWARD：表示请求在使用 RequestDispatcher forward() 时过滤。
• INCLUDE：请求在使用 RequestDispatcher include() 时过滤。
• ERROR：请求转到 error resource 时过滤。
• 以上多个值的组合

<dispatcher> 元素的使用示例，如下：

<filter-mapping>
	<filter-name>Logging Filter</filter-name>
	<url-pattern>/products/*</url-pattern>
	<dispatcher>FORWARD</dispatcher>
	<dispatcher>REQUEST</dispatcher>
</filter-mapping>

Sessions

HTTP 是一种 stateless 协议。为了构建有效的 Web 应用程序，将来自特定 client 的 request 彼此关联是必要的。Servlet specification 定义了简单的 HttpSession interface，它允许 servlet container 使用多种方法中的一种来 track 用户的 session，而无需让开发者介入方法之间的细微差别。

Session Tracking Mechanisms

Cookies

Session 通过 HTTP cookies 来跟踪会话是最常用使用的一种 session tracking mechanism 。Container 发送 cookie 给 client，client 将在接下来的访问 server 请求中发送这个 cookie，明确地将请求与会话关联。session tracking 的 cookie 的名称必须是 JSESSIONID。

URL Rewriting

URL Rewriting 是 session tracking 的另一种方法。当 client 不接受 cookie，可以使用 URL rewriting 的方式去实现 session tracking。URL rewriting 涉及到添加 session ID 到 URL path 中，container 解析该 URL，并将这个 request 与一个 session 进行关联。URL rewriting 的 URL 例子如下：

http://www.myserver.com/index.html;jsessionid=1234

只有当 client 不接受 cookie 时，URL rewriting 才会有效果。当 client 可以接受 cookie 时，URL rewiring 的 URL 不会被 container 解析，以及不会将 request 与 session 关联。

Creating Session

如果一个 session 只是一个预期的 session 而尚未建立，则认为该 session 是新的。因为 HTTP 是基于request-response 的协议，所以 HTTP session 被认为是新的，直到客户端 “join” 它为止。当 session 跟踪信息已返回到服务器以表明已建立会话时，客户端将 join 该 session。 在客户端 join session 之前，不能假定客户端的下一个请求将被识别为 session 的一部分。

Session Scope

HttpSession 对象必须是 application (or servlet context) level。不同 context 可以有相同的建立 session 的 cookie，但是 container 不能在不同的 context 之间共享这些 session 对象。

Binding Attributes into a Session

Servlet 可以通过一个 name 将一个 object attribute 绑定到 HttpSession 中。绑定在 session 中的 object 对于任何其他属于相同的 ServletContext 的 servlet 并且属于同一个会话的请求都是可以使用的 。

Session Timeouts

在 HTTP protocol 中，当 client 不在活跃时没有明确的结束信号。这意味着只有 timeout period mechanism 可以表明 client 不在活跃。

Session 的默认 timeout period 在 servlet container 中定义，它可以通过 HttpSession 接口的 getMaxInactiveInterval 方法获取。开发者可以通过 HttpSession 的 setMaxInactiveInterval 方法改变这个 timeout。这些方法的 timeout period 是以秒为单位来定义的。如果 session 的 timeout period 设置为-1，这个 session 将永远不会过期。

Important Session Semantics

Threading Issues

多个 servlet 执行请求线程可以同时访问同一个 session object。访问 session object 应该是 synchronized，开发者负责适当地 synchronizing 访问 session resources。

Client Semantics

由于 cookie 或 SSL certificates 一般是被 Web browser 控制的，它们没有与特定的浏览器窗口关联的，来自所有从 client 浏览器窗口到 servlet container 的请求可能是同一个 session。为了获得最大的可移植性，开发者应该始终假定 client 的所有窗口都参与同一个 session。

Dispatching Requests

当构建一个 Web 应用程序，将请求的处理 forward 到另一个 servlet 或 include 另一个 servlet 在 response 中的输出通常很有用。RequestDispatcher interface 提供了一种机制去实现它。

Obtaining a RequestDispatcher

获取 RequestDispatcher 接口的对象可以通过 ServletContext 接口的以下方法：

• getReqeustDispatcher
• getNameDispatcher

getRequestDispathcer 方法使用一个在 ServletContext 范围的描述路径的 String 参数。这个路径是相对 ServletContext 根目录的和以 “/” 开头的相对路径。这个方法使用这个 path 去查询一个 servlet。

Using a Request Dispatcher

使用 request dispatcher，servlet 调用 ReqeustDispatcher 接口的 include 或 forward 方法。这些方法的参数可以是传递给 javax.servlet 接口 service 方法的 request 和 response 参数或者是 request 和 response wrapper classes 的子类的实例。container 应该确保将 request 分发到目标 servlet 发生在和原始请求相同的 JVM 虚拟的同一线程。

The Include Method

RequestDispatcher 接口的 include 方法可能在任何时候被调用。include 方法的目标 servlet 可以访问 request 对象的所有方面，但是它使用 response 对象是有很多限制的。

include 的目标 servlet 仅仅可以把信息写入 response 对象的 ServletOutputStream 或者 Writer。它不能设置 header 或调用任何影响 header 的方法，调用 HttpServletRequest.getSession() 方法会抛出 IllegalStateException 异常。

Included Request Parameters

Servlet 使用 requestDispathcer 的 include 方法，以下 request attributes 是必须设置的：

• javax.servlet.include.request_uri
• javax.servlet.include.context_path
• javax.servlet.include.servlet_path
• javax.servlet.include.path_info
• javax.servlet.include.query_string

The Forward Method

调用 RequestDispatcher 的 forward 方法必须仅仅在 server 没有内容提交给给 client。如果在 response buffer 中有没有提交的输出数据，在目标的 servlet 的 service 方法调用之前，这些内容必须是清空的。如果 response 已经提交了，必须抛出 IllegalStateException。

Servlet 使用 requestDispathcer 的 forward 方法，以下 request attributes 是必须设置的：

• javax.servlet.forward.request_uri
• javax.servlet.forward.context_path
• javax.servlet.forward.servlet_path
• javax.servlet.forward.path_info
• javax.servlet.forward.query_string

Error Handling

如果 request dispatcher 的目标 servlet 抛出 runtime exception 或者 Servlet Exception or IOException checked exception，它应该传播到 calling servlet。所有其他的异常应该包装成 ServletException，并且 root cause of exception 设置为 original exception，因为不应该传播该异常。

Web Applications

Web application 是由 servlets，HTML pages，classes 和 其他资源的集合。Web 应用程序可以在不同供应商的 container 中运行。

Web Applications Within Web Servers

Web 应用程序根植于 Web server 的特定路径。例如，catalog 应用程序应该通过 http://www.mycrop.com/catalog 定位到。所有以这个前缀开头的请求都将被路由到表示 catalog 应用程序的 ServletContext。

Relationship to ServletContext

Servlet container 必须强制一个 Web 应用程序与一个 ServletContext 一一对应。ServletContext 对象为 servlet 提供了其应用程序视角。

Elements of a Web Application

一个 Web application 可能有以下内容组成：

• Servlets
• JSP Pages
• Utility Classes
• Static documents（HTML，images，sounds，etc）
• Client side Java applets，beans，and classes
• Descriptive meta information that ties all of the above elements together

Directory Structure

Web 应用程序是作为结构化目录的层次结构存在的。在应用程序的层次结构中有一个特殊的目录为 WEB-INF，这个目录包含与应用程序相关的不在应用程序的文档 root 目录中的文件。WEB-INF 节点不是应用程序的公开文档树中的一部分。容器不能将 WEB-INF 目录中包含的文件直接提供给 client。WEB-INF 目录的内容是对 ServletContext getResource 和 getResourceAsStream 方法是可见的，以及使用 RequestDispatcher 调用。如果应用开发者需要访问如应用的配置信息文件，但不希望把它直接暴露给 Client，可以把它们放到 WEB-INF 目录下。任何访问 WEB-INF 目录资源的请求将返回 404。WEB-INF 目录的内容有：

• /WEB-INF/web.xml: deployment descriptor
• /WEB-INF/classes/: servlet 和 uitlity classes.
• /WEB-INF/lib/*.jar: Java Archive files. These files contains servlets, beans, and other utility classes.

Web Application Archive File

可以使用标准的 Java archive tools 将 Web 应用程序打包并签名为 Web ARchive format (WAR) file。WAR 文件中的 META-INF 目录包含了对 Java archive tools 有用的信息。这个目录不能直接被 client 访问。

Web Application Deployment Descriptor

Web 应用程序 deployment descriptor 包含一下配置和部署信息的类型：

• ServletContext Init Paramenters
• Session Configuration
• Servlet/JSP Definitions
• Servlet/JSP Mappings
• Welcome File list
• Error Pages
• Security

Dependencies On Extensions

当大量的应用程序使用相同的 code 或 resources，它们通常将作为库文件安装在 servlet container 中。这些文件一般是常用的或标准的 API，使用它们不会牺牲可移植性。Servlet container 必须为这些 libraries 提供一个目录。位于这个目录的文件必须可用被所有 Web 应用程序使用。该目录位置是特定于 container 的。

应用程序开发者依赖的扩展必须在 WAR 文件中提供 META-INF/MANIFEST.MF entry，其中列出了 WAR 需要的所有扩展。Manifest entry 的格式应该遵循标准的 JAR manifest 格式。

Web container 必须也能够识别在 WAR 中 WEB-INF/lib 条目下的任何 library JARs 的 manifest entry 中表达的声明的依赖。

Error Handling

当异常发生在 servlet 或 JSP page，下面的属性必须设置的：

• javax.servlet.error.status_code (java.lang.Integer)
• javax.servlet.error.exception_type (java.lang.Class)
• javax.servlet.error.message (java.lang.String)
• javax.servlet.error.exception (java.lang.Throwable)
• javax.servlet.error.request_uri (java.lang.String)
• javax.servlet.error.servlet_name (java.lang.String)

这些属性允许 servlet 去生成特定的内容。

Error Pages

为了允许开发者可以去自定义 servlet 出现 error 时返回给 Web client 的内容，deployment descriptor 定义了 error page 描述的列表。这个语法允许当 servlet 或 filter 调用 HttpResponse.sendError 返回特定的 status code，或者 servlet 产生的 exception 或 error 传播到了 container 时，container 将返回资源配置。

如果在 response 上调用了 sendError 方法，container 查询使用 status-code 语法声明的 Web 应用程序的 error page 列表，并尝试进行匹配。如果匹配成功，container 返回通过 location entry 指定的资源。error page 声明的例子：

<error-page>
    <error-code>404</error-code>
    <!-- /src/main/webapp/error-404.html-->
    <location>/error-404.html</location>
</error-page>
<error-page>
    <exception-type>java.lang.Exception</exception-type>
    <location>/errorHandler</location>
</error-page>

Web Application Deployment

当一个 Web 应用程序部署到 container 中，在 Web 应用程序开始处理 client 请求之前，接下来的步骤是必须执行的：

• 实例化每一个在 deployment descriptor 中 <listener> 元素定义的 listener 的实例。
• 为了实例化实现了 ServletContextListener 的 Listener 实例，调用它的 contextInitialized() 方法。
• 实例化每个在 deployment descript 中的 <filter> 元素定义的 filter 的实例，并且调用它的 init() 方法进行初始化。
• 实例化每个包含 <load-on-startup> 的 <servlet> 元素定义的 servlet 实例，并且调用它的 init() 方法进行初始化。

Inclusion of a web.xml Deployment Descriptor

如果 Web 应用程序不包含任何 servlet，filter，或 listener 组件，这个应用程序不需要包含 web.xml。换句话说，一个仅仅包含静态文件或 JSP page 的Web 应用程序不需要出现 web.xml。

Application Lifecycle Events

应用程序 event 功能使开发者能够更好地控制 ServletContext，HttpSession 和 ServletReqeust 的生命周期，实现更好的代码分解，并提高管理 Web 应用程序使用的资源的效率。

Event Listener

应用程序的 event listener 是实现一个或多个 servlet event listener 接口的 class。在部署 Web 应用程序时，将实例化它们并将其注册在 Web container 中。

Servlet event listener 支持 ServletContext，HttpSession 和 ServletRequest 对象中状态改变的事件通知。每种事件类型有多种 listener class。开发者可以指定 container 对每种事件类型调用 listener bean 的顺序。

Event Types and Listener Interfaces

• ServletContextListener
• ServletContextAttributeListener
• HttpSessionListener
• HttpSessionAttributeListener
• HttpSessionActivationListener
• HttpSessionBindingListener
• ServletRequestListener
• ServletRequestAttributeListener

Deployment Descriptor Example

<web-app>
<display-name>MyListeningApplication</display-name>
    <listener>
    	<listener-class>com.acme.MyConnectionManager</listener-class>
    </listener>
    <listener>
    	<listener-class>com.acme.MyLoggingModule</listener-class>
    </listener>
    <servlet>
        <display-name>RegistrationServlet</display-name>
        ...etc
    </servlet>
</web-app>

Mapping Requests to Servlets

Use of URL Paths

URL path 映射规则使用下面的顺序，当成功匹配后不再继续往下匹配：

• Container 尝试查找请求路径与 servlet path 的精准匹配。
• Container 尝试循环的匹配 longest path-prefix。这是通过使用 '/' 字符作为路径分隔符来一次降低目录树的路径来完成的。
• 如果 URL path 最后部分包含扩展名，如 .jsp，servlet container 将尝试匹配处理该扩展名请求的 servlet。
• 如果上面的三个规则都没有匹配到 servlet，container 将尝试提供适合于所请求资源的内容。如果应用程序定义了默认的 servlet，则将使用它。

Specification of Mappings

在 Web 应用程序的 deployment descriptor 中，使用以下语法去定义 mappings：

• 以 / 字符开头，以 /* 字符结尾的字符串。
• 以 *. 开头作为扩展映射的字符串
• 仅包含 / 字符表明应用程序的默认 servlet。
• 其它仅仅精准匹配的字符串。

Mapping Set Example

• /foo/bar/*
• /baz/*
• /catalog
• *.bop

Security

Servlet 的 deployment descriptor 中的声明可以设置应用程序的安全。Web 应用程序包含许多用户可以访问的资源。这些资源通常是暴露在不受保护的开放网络，例如 Internet 环境中，大量的 Web 应用程序有安全性需求。Servlet container 具有满足这些要求的机制和基础结构，它们具有一下这些特征：

• Authentication: 通信实体互相证明其代表授权访问的特定身份。
• Access control for resources: 与资源进行交互的手段仅限于用户或程序的集合，以加强完整性，机密性或可用性约束。
• Data Integrity: 信息在传输的过程中不能被第三方修改。
• Confidentiality or Data Privacy: 信息仅仅对授权访问的用户可用。

Implementing Security in Servlet

在 Web 应用程序的 Deployment Descriptor 中配置 security：

• <security-role>: Defining roles.
• <login-config>: Defining how to authenticate user. e.g. login by username and password in login page.
• <security-constraint>: Defining constrained resources URI, HTTP methods, constraint role, data constraint type.

在 Apache Tomcat server 的 conf/tomcat-user.xml 文件中配置合法的角色、用户和密码。

• <role> and <user>

使用基于自定义表单或者默认的弹框方式进行安全认证。

References

[1] Java Servlet 2.5 Specification

[2] Java servlet - Wikipedia

本篇是将介绍如何更好地编程，如何编写整洁的代码，以及如何将 bad code 转换为 good code。

Clean Code

What is Clean Code

clean code 应该具有以下特点：

• 整洁性、可读性。1）简单明了，清晰地表达代码的意图。2）便于阅读，有意义的命名。3）统一的良好代码格式。
• 可维护性、可扩展性。1）单一职责，每个类和方法做好一件事。2）开闭原则。3）没有重复。4）最小化依赖。
• 健壮性。1）错误处理。2）单元测试完全覆盖，无潜在 bug。
• 高效性。性能尽可能最优。

Why do We Need Clean Code

代码是需求的最终表达形式，实现一个软件我们需要编写正确、整洁的代码，使其高效地在机器上运行，以及便于扩展和维护。

编写 bad code 可能会导致严重的后果。bad code 会使得团队的生产力持续降低。bad code 可能导致软件无法维护，以及存在大量 bug，最终导致软件无法正常运行。最后，不得不重新设计代码，建立一个新的团队重构旧的系统，以及跟进旧系统的改变。

产生 bad code 的原因

产生 bad code 的原因有很多，如：1）需求的改变。2）时间规划太紧，没有足够的时间去做得更好。3）对一个程序很疲倦，想要早点结束。4）手上堆积了很多其它的事，想要赶紧做完它，然后做其它事情。5）管理者的管理不当。…

作为程序开发者，我们不能一味地抱怨外部原因，我们更应该反思我们自己，我们能否做得更好。1）当软件的时间规划和安排不合理时，我们应该及时的反馈我们的想法。大部分管理者想要看到事实，以及想要 good code。2）在开发过程中，我们应该保持专业的态度，持续保证代码的整洁，就像医生做手术前要洗手一样，多花一点时间保持代码的整洁，让保证代码不会变成 bad code。唯一保证 deadline 的方法就是在任何时候尽可能地保持代码地整洁，为了加快速度编写混乱的代码最终会拖垮你的进度。

Meaningful Names

命名在软件开发中无处不在。我们需要为变量、方法、参数、类、包和项目文件目录等命名。我们需要大量的命名，因此我们应该把它做得更好。这一部分，我们将介绍好的命名规则，以及尽力避免的不好的命名规则。

Rules for Creating Good Names

选择一个好的名称是需要时间的，但它节省的时间大于它的花费。持续关注代码中名称，当你发现更好的名称时立刻改变它。

Use Intention-Revealing Names

一个好的名称应该告诉我们它为什么存在，它能做什么，它如何使用。如果一个名称需要注释说明，然么这个名称没有展现它的意图。使用揭示意图的名称更容易理解和改变代码。示例如下：

错误写法

int d; // elapsed time in days

正确写法

int elapsedTimeInDays;
int daysSinceCretation;

揭示意图常用的方法：

• 使用揭示意图的名称。
• 使用静态变量代替直接量。
• 使用对象封装一组数据。

Make Meaningful Distinction

如果名称必须不同，那么它们一定存在某些不同。有意义的区分不同的名称，可以让读者清晰地知道它们之间的不同。

常见无意义区分：

• 使用数字序列命令 a1, a2, .. aN。如 copyChars(char a1[], char a2[]) => copyChars(char source[], char destination[])
• 使用同义词。如 1）info 和 data，ProductInfo 和 Productdata。2）a, an, the，zork 和 theZork。

Use Pronounceable Names

我们大脑更习惯处理可发音的语言。可发音的名称更容易与他人交流和讨论。命名时尽量使用完整的单词的组合。示例如下：

错误的写法

class DtaRcrd102{
    private Date genymdhms;
    private Date modymdhms;
    private final String pszqint = "102";
}

正确的写法

class Customer {
    private Date generationTimestamp;
    private Date modificationTimestamp;
    private final String recordId = "102";
}

Use Searchable Names

单字母的名称和数字常量等名称很难在文本中定位。尽量不要使用单字母命名和直接使用常量。

Class Names

Classes 和 objects 的名称应该为名词或名词短语。如 Customer，WikiPage，Account，AddressParser 等。

Method Names

方法名称应该为动词或者动词短语。如 postPayment，deletePage，save 等。

Pick One Word per Concept

为一个抽象概念选择一个词，并且坚持使用它。例如，fetch，retrieve，get 等相似含义的名称用在不同的类中功能相似的方法中，会使人感到困惑。类似还有 controller，manager 和 driver。同一个概念应该坚持使用同一个单词。

Use Solution Domain Names and Problem Domain Names

阅读代码的人一定是程序员，我们应该坚持使用计算机术语。计算机术语大部分程序员都比较熟悉，遇到不清楚的术语，可以在网上快速查询得到结果，不必与作者沟通。

Add Meaningful Context

增加有意义的 Context 让读者能清晰直观地知道这个变量是某个类的一部分。

如 firstName，lastName，street，houseNumber，city，state，zipcode 等变量是 Address 类中的成员。其中 state 不能直观的看出它是 adderss 的一部分。我们可以添加前缀来增加 context。state => addressState。

Avoid Creating Bad Names

Avoid Disinformation

我们应该避免留下误解代码含义的错误线索。

• 避免单词缩写。如， hypotenuse 缩写成 hp。
• 集合对象不确定数据结构时，应避免添加固定的类型后缀。如，accountList => accountGroup, bunchOfAccounts, or accounts.

Avoid Encodings

编码的名称增加解码的负担。编码名称一般很少是可发音的，以及它容易误解类型。尽量避免使用编码的名称。

常见的编码命名方式：

• 匈牙利表示法（Hungarian Notation）变量名以一个或多个小写字母开头，代表变量的类型。后面附以变量的名字。这种方案允许每个变量都附有表示变量类型的信息。如 int[] aNames，boolean bDeleted
• 成员前缀 m_。
• 接口和实现。使用 Ixxx 表示接口,如 IShapeFactory。

Avoid Mental Mapping

避免思维映射，即避免使读者将名称翻译为它们已经知道的概念。如设计模式名称、算法名称等等。在命名时我们使用问题领域术语和解决方案领域术语。

Don’t Be Cute

如果名称太机灵，它们将仅能被作者分享过它的含义的人记住。如 HolyHandGrenade 实际上表示 DeleteItems，whack() 实际表示 kill()，eatMyShorts() 表示 abort()。 尽量避免这种命名方式。

Don’t Pun

避免使用相同的词表示两个目的。使用相同的术语表示两个不同的想法是一语双关的，让人不易理解，产生误解。如，一个 add 方法用于将两个值相加或连接。然而，另一个 add 方法则表示将元素添加到集合对象中。我们不应该一词双关，我们可以使用 insert 或 append 代替 add。

Don’t Add Gratuitous Context

不要增加不必要的 context。如，为某个模块的所有 classes 添加前缀标识。

有意义的命名总结

单个命名应该：揭示意图，可发音，可搜索，一致性，使用专业术语，增加有意义的 context。

多个命名之间应该：可区分。

有意义的命名的实现目标：1）整洁性、可维护性。名称揭示意图，代码解释一切，无注释。2）可读性。代码命名简单、明确，让人更轻松、更快地阅读和理解代码。

Functions

Small

function 的第一规则：它们应该非常小。

1. Blocks and Indenting。if..else，while 等代码块应该占一行，作为方法调用。代码块作为方法调用可以有一个很好的描述名称。

2. indent level 不应该超过一层或两层。这使得 functions 更容易阅读和理解。

Do One Thing

Functions 应该做一件事，它们应该做好，它们应该只做一件事。

如何知道是否是只做一件事：

• function 只有一层抽象，即不包含多个不同的抽象层级。
• function 不能被划分多个 sections。

Switch Statements

让 switch 语句只做一件事是困难的，根据它的性质，switch 语句总是做 N 件事。我们可以将 switch 的每一个分支封装成一个方法。

Use Descriptive Names

functions 的名称应该描述 function 做了什么。不要害怕使得名称很长，一个长的 descriptive name 比短的神秘的名称更好。一个长的 descriptive name 比一个长的 comment 更好。

Function Arguments

理想的 function 参数的个数是0个，其次是1个和2个，3个参数是尽量避免的。

多个参数的问题：

• 参数需要占用额外的概念理解的精力，每次都需要解释它。
• 参数越多越难测试。

其它参数问题

1. Flag 参数。传递一个 boolean 参数给方法是很糟糕的，你明显的表现出这个 function 不是做一件事。
2. 参数对象。超过两个或三个参数应该封装成一个 class。
3. 动词和关键词。方法名应该是动词+名词对。如使用 writeField() 代替 write()。
4. 输出参数。输出参数比输入参数是更难理解的。我们既要关注输入是什么，也要关注输出是什么，它使我们花费双倍的精力。如果你的 function 必须改变某些状态，让它改变其拥有对象的状态。如 appendFooter(String report) => report.appendFooter()。

Have No Side Effects

你的 function 保证了做一件事，但可能存在其它隐藏的事情。示例如下：

public boolean checkPassword(String userName, String password){
    User user = UserGateway.findByName(userName);
    if (user != User.NULL){
        if (user.getPassword.equals(password)){
            Session.initialize();
            return ture;
        }
    }
    return false;
}

上面的 checkPassword() 方法中的 Session.initialize() 是隐藏的事情。因为初始化会话这件事不属于检查密码。可以将方法改为 checkPasswordAndInitializeSession()，其实现如下：

public boolean checkPasswordAndInitializeSession(String userName, String password){
    boolean passwordRight = checkPassword(userName, password);
    if (passwordRight){
        Session.initialize();
        return true;
    }
    return false;
}

Command Query Separation

Functions 应该是做某些事或者回答某些事，而不是两者都做。你的 function 应该改变一个对象的状态或者返回一个对象的一些信息。一个错误的例子如下：

public boolean set(String attribute, String value);

上面这个方法即设置了对象的属性，又返回操作是否成功和属性是否存在。这会导致出现奇怪的语句，如 if (set("username", "unclebob"))。

上面的方法应该划分为两个方法，一个执行操作，一个查询状态。如下：

public void set(String attribute, String value);
public boolean attributeExists(String attribute);

Prefer Exception to Returning Error Codes

执行命令的方法返回 error codes 轻微地违反了 command query separation 原则。当你返回一个 error code，调用者必须立刻处理 error，才能继续执行其它语句，这将导致代码很多层嵌套判断。示例代码如下：

if (deletePage(page) == E_OK){
    if (registry.deleteReference(page.name) == E_OK){
        if (configKeys.deleteKey(page.name.makeKey()) == E_OK){
            ...
        } else {
            
        }
    }
}

你可以使用 exception 代替返回 error codes，这样的话，不需要立即处理错误情况，代码没有太多的嵌套，是代码更整洁。代码如下：

try {
    deletePage(page);
    registry.deleteReference(page.name);
    configKeys.deleteKey(page.name.makeKey());
}
catch (Exception e) {
	logger.log(e.getMessage());
}

Extract Try/Catch Blocks

Try/catch 代码块是丑陋的，它混合了正常代码和错误处理代码。更好的办法是将 try/catch 代码块封装到一个独立的方法。上面的代码可以改为：

public void delete(Page page){
    try {
        deletePageAndAllReference(page);
    } catch (Exception e){
        logError(e);
    }
}
public void deletePageAndAllReferences(Page page) throws Exception {
    deletePage(page);
    registry.deleteReference(page.name);
    configKeys.deleteKey(page.name.makeKey());
}

private void logError(Exception e){
    logger.log(e.getMessage());
}

Error Handling Is One Thing

Functions 应该做一件事，Error handling 它是一件事，所以，一个处理 errors 的 function 不应该做其它的事。即在 catch/finally 代码块后面不应有其它的语句，或一个处理 error 的 function 仅包含 try/catch/finally 代码块。

Don’t Repeat Yourself

在软件中重复是所有罪恶的根源。我们应该尽可能的消除重复的代码。

Clean functions 的总结

一个 function 应该尽量小，只做一件事，尽量少的参数，和使用描述性的名称。

如何只做一件事：只有一层抽象，不能划分为多个 sections，没有隐藏功能，命令和查询分离。

其它问题：错误处理，消除重复。

一个专业的程序员把系统当作故事来讲诉，而不是编写程序。

Comments

Don’t comment bad code–rewrite it.

注释的正确用法是为了弥补我们在代码中失败地表达自己。当你需要写注释时，你需要思考它有没有方式在代码中表达。每次你写了一个注释，你应该感到你表达能力的失败。

为什么尽量不写注释，因为程序员很少去维护注释的正确性。

• 注释不能弥补差的代码。与其花时间去写注释解释混乱的代码，不如花时间使代码整洁。

• 在代码中解释你自己。在很多时候可以创建一个方法来代替你注释要说明的。如下：

  // Check to see if the employee is eligible for full benefits if ((employee.flog & HOURLY_FLAG) && (employee.age > 65))

  代替为

  if (employee.isEligibleForFullBenefits())

Good Comments

有些注释时有必要的或有益的。始终记住真正好的注释是找到一种方式不写注释。

常见的有必要的注释

• Legal Comments。版权和作者声明是有必要的和合理的。如 // Copyright (C) 2003,2004,2005 by Object Mentor, Inc. All rights reserved.
• Explanation of Intent。解释为什么这么做。如 // we are greater because we are the right type.
• Warning of Consequence。警告注释。如 // Don't run unless you, have some time to kill
• TODO Comments。TODO 一般用于此刻没时间做或者暂时做不了的事。它解释代码的问题和未来应该做什么。如 //TODO-MdM these are not needed // We expect this to go away when we do the checkout model
• Amplification。详述注释。用于详述某些重要的事情，其他人可能认为不重要的事。// the trim is real important. It removes the starting spaces that could cause the item to be recognized as another list.

Formatting

Code formatting 是重要的，编码风格和可读性会持续影响软件的维护性和扩展性。

Vertical Formatting

1. 行数。一个源码文件不应该超过 500 行，一般最好 200 行左右。小文件通常比大文件更容易理解。
2. 自顶向下阅读。最上面的方法提供高层次的概念和算法。细节应该是从上到下递增的。
3. 垂直的间隔。不同概念的一组代码应该用空行分隔。
4. 垂直的密度。垂直密度暗示了紧密的关系，相关性高得代码应该紧密排布在一起。
5. 垂直的距离。紧密关联的概念应该保持垂直方向接近的。1）变量声明语句应该尽可能接近它们使用的地方。2）实例变量应该声明在 class 的顶端。3）依赖的方法。一个方法调用另一个方法，它们应该在垂直方向相互接近的，调用者应该在被调者的上面。4）概念上类似。相似的内容应该在垂直方向相互接近的。如方法的重载，相同的方法名，不同的参数的方法，它们的功能是类似的。
6. 垂直的顺序。方法的调用依赖时指向下面方向的。

Horizontal Formatting

1. 水平的字符数应该不超过 120 个字符。
2. 水平的间隔和密度。操作符与操作数之间有一个空格，将它们分成左边和右边两个部分，看起来更清晰。
3. 缩进。使用缩进来表示代码语句范围的层级结构。

Team Rules

每一个程序员都有自己最喜欢的格式规则，但是如果在一个团队中工作，我们需要按照团队的规则去编写代码。一个团队的所有开发者应该同意一个编码风格，然后每个团队成员应该使用这个风格。我们需要一个统一的编码风格。

Error Handling

错误处理时必须要做的一件事。输入可能异常，设备可能失败，总的来说，事情可能发生错误，程序员有责任确保当发生异常时系统可以正常工作。

1. 使用异常处理而不是返回值。使用异常处理比返回错误码更整洁。返回错误码需要立刻处理，会导致多层嵌套的复杂结构。
2. 尝试编写强制异常的测试。
3. 使用 Unchecked Exceptions。Checked exceptions 违反了 Open/Closed 原则。如果你抛出一个 checked exception，你需要在你 catch 和 throw 之间的所有方法上声明异常。
4. 根据调用者的需求定义异常类。我们定义异常类最重要是它们是怎么被捕获的。通常单个异常是较好的。我们可以简化 API 将多个异常包装成一个异常。
5. 不要用异常处理 try/catch 来做业务流程处理。即 catch 代码块中不应该出现业务代码。
6. 不要返回 Null。返回 null，给调用者造成额外的负担，null 会导致很多 null 检查代码。如果尝试返回 null 时，可以考虑抛出异常或者返回特殊对象代替。如 Collections.emptyList()。
7. 不要传递 Null 作为参数。

Unit Test

单元测试保证了代码的每一个功能都是我们所期待的结果。

Keeping Tests Clean

混乱的测试代码比没有测试更糟糕。测试代码必须随着生产代码的发展而改变，混乱的测试代码很难去改变，测试代码越来越多，越来越混乱，最终整个测试代码会被弃用。

没有测试代码，不能保证代码的改变是正常工作的，不能保证改变系统的一部分没有破环系统的其他部分。所以系统的缺陷率开始上升。他们停止清洁和整理生产代码，因为他们害怕改变会影响系统正常工作。他们的生产代码开始腐烂变坏。

测试代码和生产代码是一样重要的。它需要思考，设计和关注。它必须保持和生产代码一样整洁。

单元测试让我们的代码是灵活的、可维护的和可重用的。因为如果你有测试，你就不害怕改变代码，你可以不断地优化你的代码。

Clean Tests

使测试代码整洁的方法：可读性。让测试方法和正常方法一样整洁，使它尽量的小，只做一件事，只有一层抽象，和使用描述性的名称等。

Single Concept per Test

每个测试方法应该只测试单个概念。

F.I.R.S.T

编写整洁测试代码的 5 个规则：

• Fast。测试应该是快的。如果测试运行很慢，你不想频繁地运行它们。如果你没有频繁地运行测试，你不会更早地发现问题，以及更轻松地修复问题。
• Indenpent。测试方法应该不依靠其它代码。你应该能够独立地测试，以及以任何顺序运行测试。
• Repeatable。测试应该在任何环境重复执行。如生产环境、QA 环境，和本地环境等。
• Self-Validating。测试应该有一个布尔输出。你可以不通过日志文件知道测试是否通过。
• Timely 。测试代码应该在业务代码之前写。

Classes

Class Organization

标准的 Java convention，一个 class 由一组变量开始，首先是 pubilc static constants，然后是 private static 变量，最后是 instance 变量。变量后面是方法，方法按照抽象层次由高往下。

Class Should Be Samll

class 的第一规则是 class should be small。class 的名称应该说明其应履行的职责。我们应该能够不使用 “if”, “and”, “or”, “but” 等单词情况下，用 25 words 来写出 class 的简单的描述。

Single Responsibility Principle 表示一个 class 或 module 应该只有一个发生改变的原因。这个原则帮助我们进行职责的定义和对 class size 的指导。尝试识别职责（reasons to change）常常能帮助我们在代码中找到和创建更好的抽象。

Cohension。classes 应该有少量的实例变量。一般更多的变量被一个方法操作，这个方法对 class 来说越内聚。当一个 classes 不内聚，应该分离它们。

Organizing for Change

大部分系统是不断改变的。每一个改变都可能使得系统的其它部分不能正常工作的风险。我们可以阻止我们的 classes 减少改变的风险。下面这些原则可以帮助我们更好地组织 classes 以减少代码改变带来的风险：SRP（Single Responsibility Principle），Open-Closed Principle（OCP），Dependency Inversion Principle（DIP）

Summary

Most Important

• Meaningful Names
• SRP（Single Responsibility Principle）

References

[1] Clean Code by Robert C. Martin

设计模式的总结

设计模式的用途

设计模式常见的用途如下：

• 可复用性：减少冗余代码。
• 可扩展性：隔离职责，松耦合。很容易添加新类型的 class，容易添加新的功能。
• 灵活性：方便修改功能，少改动，独立修改。方便控制管理对象。
• 易用性：使得 Client 操作简单。

深入理解设计模式系列文章

• 深入理解设计模式（一）：介绍设计模式
• 深入理解设计模式（二）：创建型模式
• 深入理解设计模式（三）：结构型模式
• 深入理解设计模式（四）：行为型模式
• 深入理解设计模式（五）：设计模式的总结

本篇将介绍设计模式中常见的11种行为型模式，其中设计模式的实现代码使用 Java 语言描述。

Behavioral Patterns

行为型模式它关注对象之间的算法和职责分配。它不仅是描述对象和类的模式，也是描述它们之间交流的模式。

Chain of Responsibility

What

通过给多个对象处理请求的机会，避免耦合请求发送者与接收者。Chain 接收请求，并沿着 chain 传递请求直到有一个对象能够处理该请求为止。

Why

Motivation

避免将请求者与具体的接收者绑定。使用一个类统一接收所有请求，接收者连接成一条链来处理请求。

Applicability

• 超过一个对象去处理请求。
• 你想发送一个请求给一些对象中的一个，并且不明确指定接收者。
• 动态指定一组对象处理一个请求。

Solution

Structure

类结构

对象结构

Participants

• Handler：定义一个接口去处理请求。
• ConcreteHandler：处理请求。接收继任者（successor）。如果它能处理一个请求，则处理它；如果不能，则将这个请求转发给它的继任者。
• Client：初始化一个请求，发送给 chain 上的一个 ConcreteHandler 对象。

Collaborations

• 当 Client 发送一个请求，这个请求在 Chain 上传播，直到有一个 ConcreteHandler 对象能够处理它。

Implementations

public abstract class Handler{
    private Handler nextHandler;
    
    public abstract void handleRequest();
    
    public void setNext(Handler handler){
        this.nextHandler = handler;
    }
}

public class ConcreteHandler1 extends Handler{
    public void handleRequest(int request){
        if (request == 1){
            System.out.println("handle by ConcreteHandler1");
        }else{
            if (nextHandler != null)
                nextHandler.process(request);
        }
    }
}

public class ConcreteHandler2 extends Handler{
    public void handleRequest(int request){
        if (request == 2){
            System.out.println("handle by ConcreteHandler2");
        }else{
            if (nextHandler != null)
                nextHandler.process(request);
        }
    }
}

public class Client{
    public static void main(String[] args){
        Handler handler = new ConcreteHandler1();
        Handler nextHandler = new ConcreteHandler2();
        handler.setNext(nextHandler);
        int request1 = 1, request2 = 2;
        handler.handleRequest(request1);
        handler.handleRequest(request2);
    }
}

Consequences

Benefits

• 减少了耦合。
• 增加了给对象分配职责的灵活性。

Drawbacks

• 请求的接收不能保证。因为一个请求没有明确指定接收者，所有不能保证它能被处理。

Command

What

将一个请求封装为一个对象，因此让你参数化客户端的不同请求。

Why

Motivation

有时需要发送一个请求给对象，但不知道任何关于请求的操作或者请求的接收者。

Applicability

• 参数化执行动作，将其封装为对象。
• 在不同的时间指定，排队和执行请求。Command 对象的生命周期可以独立于原始请求。
• 支持 undo。Command 执行操作可以存储状态用来反转影响 command 本身。
• 支持 logging changes。所以可以在系统崩溃后可以重新正确运行。
• 围绕基于 primitive 操作的 high-level 操作来构建系统。

Solution

Structure

Participants

• Command：声明用于执行操作的接口。
• ConcreteCommand：定义 Receiver object 和 action 之间的绑定。通过调用接收者相应的操作来实现 Execute。
• Cient：创建一个 ConcreteCommnd 对象，以及设置它的 receiver。
• Invoker：请求 command 得到请求结果。
• Receiver：知道如何执行与请求相关的操作。

Collaborations

• Client 创建一个 ConcreteCommand 以及指定它的 receiver。
• Invoker 对象存储 ConcreteCommand 对象。
• Invoker 通过调用 command 对象的 execute() 方法来发出请求。
• ConcreteCommand 对象调用接收者的操作来完成请求。

Implementations

public interface Command{
    void execute();
}
public class Receiver{
    public void action1(){
        System.out.println("action 1 executing...");
    }
    public void action2(){
        System.out.println("action 2 executing...");
    }
}
public class ConcreteCommand1 implements Command{
    public Receiver receiver;
    public ConcreteCommand1(Receiver receiver){
        this.receiver = receiver;
    }
    public void execute(){
        receiver.action1();
    }
}
public class ConcreteCommand2 implements Command{
    public Receiver receiver;
    public ConcreteCommand1(Receiver receiver){
        this.receiver = receiver;
    }
    public void execute(){
        receiver.action2();
    }
}
public class Invoker{
    Command slot;
    public void setCommand(Command command){
        this.slot = command;
    }
    public void requestExecute(){
        this.slot.execute();
    }
}
public class Clinet{
    Invoker invoker = new Invoker();
    Receiver receiver = new Receiver();
    invoker.setCommnad(new ConcreteComand1(receiver));
    invoker.requestExecute();
    invoker.setCommnad(new ConcreteComand2(receiver));
    invoker.requestExecute();
}

Consequences

Benefits

• Command 将调用操作的对象与知道如何执行该操作的对象分离。
• 你可以装配命令作为 composite 命令。
• 很容易添加新的 Command。因为你不需要该改变存在的 classes。

Interpreter

What

给定一种语言，定义其语法表示形式，以及使用该表示形式来解释语言中的句子。

Why

Motivation

如果一个种特殊的问题经常发生，它可能值得用简单的语言将问题的实例表达为句子。然后，你可以构建 interpreter 通过解释这些句子来解决问题。

Applicability

• 语法是简单的。
• 效率不是一个关键问题。

Solution

Structure

Participants

• AbstractExpression：定义一个抽象 interpret 操作，它存在于所有 abstract syntax tree 中的节点。
• TerminalExpression：实现与 terminal symbols 有关的 interpret 操作。
• NonterminalExpression：实现 nonterminal symbols 相关的 interpret 操作。
• Context：包含给 interpreter 的全部信息。
• Client：构建一个抽象的 syntax tree 表示一个符合语法规定的特定的句子。调用 interpret 操作。