quarks.oplet.plumbing

Class PressureReliever<T,K>

  • Type Parameters:
    T - Tuple type.
    K - Key type.
    All Implemented Interfaces:
    java.io.Serializable, java.lang.AutoCloseable, Consumer<T>, Oplet<T,T>


    public class PressureReliever<T,K>
    extends Pipe<T,T>
    Relieve pressure on upstream oplets by discarding tuples. This oplet ensures that upstream processing is not constrained by any delay in downstream processing, for example by a sink oplet not being able to connect to its external system. When downstream processing cannot keep up with the input rate this oplet maintains a defined window of the most recent tuples and discards any earlier tuples using arrival order.

    A window partition is maintained for each key seen on the input stream. Any tuple arriving on the input stream is inserted into the window. Asynchronously tuples are taken from the window using FIFO and submitted downstream. The submission of tuples maintains order within a partition but not across partitions.

    Tuples are discarded and not submitted to the output port if the downstream processing cannot keep up the incoming tuple rate.

    • For a count PressureReliever up to last (most recent) N tuples are maintained in a window partition.
      Asynchronous tuple submission removes the last (oldest) tuple in the partition before submitting it downstream.
      If when an input tuple is processed the window partition contains N tuples, then the first (oldest) tuple in the partition is discarded before the input tuple is inserted into the window.


    Insertion of the oplet into a stream disconnects the upstream processing from the downstream processing, so that downstream processing is executed on a different thread to the thread that processed the input tuple.

    See Also:
    Serialized Form
    • Constructor Detail

      • PressureReliever

        public PressureReliever(int count,
                                Function<T,K> keyFunction)
        Pressure reliever that maintains up to count most recent tuples per key.
        Parameters:
        count - Number of tuples to maintain where downstream processing cannot keep up.
        keyFunction - Key function for tuples.
    • Method Detail

      • accept

        public void accept(T tuple)
        Description copied from interface: Consumer
        Apply the function to value.
        Parameters:
        tuple - Value function is applied to.
      • close

        public void close()
                   throws java.lang.Exception
        Throws:
        java.lang.Exception

Copyright IBM 2015,2016 - 2f6ad0e-20160307-0902