Abstract: Hydraulic steel structures (HSS) are components of navigation, flood control, and hydropower projects that control or regulate the flow of water. Damage accumulates in HSS as they are operated over time, and they must be inspected periodically. This is often accomplished using nondestructive testing (NDT) techniques. If damage is detected, the structure’s fitness for continued service must be evaluated, which requires information on the location and size of discontinuities. This information can be obtained using ultrasonic testing (UT) techniques. However, there is limited information on the reliability of UT techniques with respect to detecting, sizing, and characterizing flaws in HSS. This study addresses this gap. Round-robin experiments were carried out using phased-array ultrasonic testing (PAUT) to scan weld specimens representing a variety of HSS geometries. The results of the round-robin experiments were analyzed to estimate the probability of detection (POD) and to assess the influence of factors potentially affecting POD. Uncertainty in estimates of flaw length and height were described, and partial safety factors were derived for use in fitness-for-service analyses. These results demonstrate the importance of the technician as a factor influencing the reliability of NDT techniques applied to HSS.