The possibility of actually implementing a useful GPU trigger algorithm ultimately depends on the possibility of running it at a high input rate, and within a time lower than the predefined trigger latency.\\

Before analysing the real time performance of the trigger, it is useful to remind that all GPU-related latencies depend on the specifications of the device on which the program is compiled and executed. Appendix~\ref{app:devicequery} lists all the relevant features of the NVIDIA Tesla K20 GPU that was used for the tests reported here. In particular, this device has a warp size of 32 threads. This means that, if we set a number of threads per block equal to the maximum hit multiplicity, those data sets with hit multiplicity $n_{hit} \leq 32$ will be processed with a single warp execution, while the $5 \leq n_{hit} \leq 64$ data sets will require a queue of two warps for each triplet (triplets are processed concurrently, thanks to the high number of available CUDA cores).\\

\begin{figure}
\centering
\includegraphics[height=0.3\textheight]{Images/pdf/gpu_algo/tests/time_h2d}
%\vfill
\includegraphics[height=0.3\textheight]{Images/pdf/gpu_algo/tests/time_ker}
%\vfill
\includegraphics[height=0.3\textheight]{Images/pdf/gpu_algo/tests/time_d2h}
\caption{GPU execution latencies for a batch of 256 $\pi^+\pi^0$ events preliminarily filtered by the standard L0 selection. The measurement was repeated 100 times. The first panel shows the time needed to copy the whole data set from the host RAM to the global memory of the GPU. The second panel shows the kernel execution time. The last one shows the time needed to copy the results back to the host.}\label{img:alltimes}
\end{figure}

The histograms of Figure~\ref{img:alltimes} report separately the three CUDA latencies adding up to the total latency of the GPU trigger:
\begin{enumerate}
\item the time $\Delta t_{H\rightarrow D}$ needed to copy the data corresponding to a batch of events into the global memory of the GPU;
\item the kernel execution time $\Delta t_{kernel}$;
\item the time $\Delta t_{D\rightarrow H}$ needed to copy the results back to the host user space (an eventually to the TDAQ system).
\end{enumerate}
The measurements were repeated 100 times, using the same L0-filtered $\pi^+\pi^0$ data set (256 events). The measured latencies are:
\begin{align}
&\Delta t_{H\rightarrow D} = 27 \pm 2~\si{\micro\second}\\
&\Delta t_{kernel} = 196 \pm 2~\si{\micro\second}\\
&\Delta t_{D\rightarrow H} = 24 \pm 2~\si{\micro\second}
\end{align}
summing up to a total latency $\Delta t_{tot}$ (\figurename~\ref{img:timetot})
\begin{align}
&\Delta t_{tot} = 247 \pm 6~\si{\micro\second}
\end{align}
and therefore to a total latency \textit{per event}:
\begin{align}
&\Delta t_{evt} = 0.97 \pm 0.02~\si{\micro\second}\label{evttime}
\end{align}
since 256 events are processed per batch.\\

These simulated events were preloaded in an internal emulation buffer of the TEL62 TDAQ boards and sent from there to the NIC at 600$\pm$25~\si{\kilo\hertz} rate. Due to the limited buffer memory of the TEL62 board, it was not possible to prepare large data files with more than 256 events; nor was it possible to transmit different samples in a queue. Therefore, in order to measure the time response stability of the trigger, we had to capture\footnote{The Wireshark network protocol analyser (\url{http://www.wireshark.org/}) was used for this purpose.} a set of UDP packets (256 events) and transmit them in a loop from an Ethernet interface to another one on the same PC, using the \texttt{pfsend} application provided with the DNA driver APIs\footnote{\url{http://www.ntop.org/solutions/wire-speed-traffic-generation/}}. In the future, it will be useful to test the framework with a continuous stream of real data from TEL62 boards.\\

\begin{figure}
\centering
\subfigure[Total GPU trigger latency for a batch of 256 events]{\includegraphics[width=\textwidth]{Images/pdf/gpu_algo/tests/time_tot}\label{img:timetot}}
\vfill
\subfigure[Total GPU latency as a function of the number of iterations.]{\includegraphics[width=\textwidth]{Images/pdf/gpu_algo/tests/time_stability}\label{img:stability}}
\caption{Total trigger execution time for 256 L0-filtered $\pi^+\pi^0$ and 256 unfiltered $\pi^+\pi^+\pi^-$ events (top panel). The measurement was repeated 100 times. The bottom plot shows the same quantity as a function of the number of iteration, and for all the data sets used for the tests discussed in Section~\ref{ch:results}.}\label{img:timetot-stability}
\end{figure}

\figurename~\ref{img:stability} reports the results of some time stability tests. The total execution time was analysed using all the data sets discussed in Section~\ref{ch:results}. Here, the total GPU latency is reported as a function of the number of the processed batch: 100 batches = 25600 events were transmitted to the NIC in a row, with event packets sent at a rate of 0.1~\si{\giga\byte/\second}, that corresponds to about 0.7 to 5.6~\si{\mega\hertz} event rate depending on the number of hits. These results show that the GPU performs best when it is under full load, i.e. the maximum execution speed is not reached for the first and last few executions. However, this should not be a problem if the trigger operates with a stable particle beam.\\

From Figures~\ref{img:timetot} and~\ref{img:stability} we can also infer that the \textit{plateau} latency of the trigger is not doubled when the number of threads per block is raised from 32 to 64 ($5 \leq n_{hit} \leq 64$ event sets). In fact, the total kernel execution time results from the sum of two contributions: a kernel launch time, that is fixed and not measurable, and the time needed to perform the operations issued in the kernel function. For example, the total time needed to process 256 $3\pi$ events, with hit multiplicity $5 \leq n_{hit} \leq 64$, amounts to
\begin{align}
&\Delta t_{tot} = 271 \pm 5~\si{\micro\second}
\end{align}
corresponding to
\begin{align}
&\Delta t_{evt} = 1.06 \pm 0.02~\si{\micro\second}\label{evttime-2}
\end{align}
per event, to be compared with the result reported above in Eqn.~\ref{evttime}, that was achieved by running the kernel function on 32 threads per block.\\

The above reported timing tests are encouraging. In particular, from the results in Eqns.~\ref{evttime} and~\ref{evttime-2} we learn that 10 Tesla K20-equivalent GPUs would be enough to handle a data input rate of 10~\si{\mega\hertz}. A ``triggerless'' approach using normal processors would require a much more larger and expensive PC farm.