HIVE-28265: Fix JDBC timeout message for hive.query.timeout.seconds#6412
HIVE-28265: Fix JDBC timeout message for hive.query.timeout.seconds#6412ashniku wants to merge 5 commits intoapache:masterfrom
Conversation
Use server-side HiveSQLException with effective timeout seconds before cancel(TIMEDOUT) so GetOperationStatus exposes correct errorMessage. JDBC: prefer server message; ignore bogus 'after 0 seconds'; fall back to Statement queryTimeout or last SET hive.query.timeout.seconds tracked on HiveConnection. Parse SET assignments anywhere in SQL (last wins). SQLOperation async: do not overwrite operationException when already TIMEDOUT. Tests: TestJdbcDriver2 (session SET + sleep UDF; tighten testQueryTimeout). Made-with: Cursor
…meout in tests - Extract TIMEDOUT/CANCELED handling into helpers to satisfy Sonar (complexity, nested blocks, nested ternary). - Add @after in TestJdbcDriver2 to run SET hive.query.timeout.seconds=0s on the shared connection so testQueryTimeoutMessageUsesHiveConf does not leave a 1s server-side limit for other tests (fixes Jenkins SQLTimeoutException cascades). Made-with: Cursor
asf-ci-hive
added
tests pending
tests unstable
and removed
tests failed
tests pending
tests unstable
labels
asf-ci-hive
added
tests pending
tests unstable
and removed
tests unstable
tests pending
labels
| fail("Expecting SQLTimeoutException"); | ||
| } catch (SQLTimeoutException e) { | ||
| assertNotNull(e); | ||
| assertTrue("Message should reflect JDBC query timeout (1s): " + e.getMessage(), |
There was a problem hiding this comment.
Can you show us an example about the whole output that demonstrates the change?
I wonder if it is possible to get any number other than the timeout in the message. Like a timestamp or maybe a query id, host name, etc. Asserting to a single number in a string looks a little bit fragile to me.
There was a problem hiding this comment.
Introduced constant QUERY_TIMED_OUT_AFTER_1_SECONDS = "Query timed out after 1 seconds" with Javadoc that this is the full message from HS2 / client (no query id, host, timestamp in that string for these paths).
testQueryTimeout now uses assertEquals, expected value = that constant, with a failure message that repeats the example text.
Use a regex for 'timed out after 1 seconds' instead of contains("1") to
avoid false positives (e.g. 10-second timeouts).
Made-with: Cursor
| * {@code N == 1} with flexible whitespace so we do not treat {@code 10} or unrelated digits as {@code 1}. | ||
| */ | ||
| private static boolean isQueryTimedOutAfterOneSecondMessage(String msg) { | ||
| return msg != null && msg.matches("(?is).*timed out after\\s+1\\s+seconds.*"); |
There was a problem hiding this comment.
We know it is 1 sec. And we don't accept any other output in that case.
In my opinion, regex here can be a little bit overkill.
What about something like:
final String expectedMessage = "Query timed out after 1 seconds";
assertEquals("Message should reflect JDBC query timeout", expectedMesage, message);
There was a problem hiding this comment.
Removed isQueryTimedOutAfterOneSecondMessage (regex helper).
Positive assertions use assertEquals("…", QUERY_TIMED_OUT_AFTER_1_SECONDS, e.getMessage()) in both timeout-related tests.
| assertNotNull(e); | ||
| assertTrue("Message should reflect JDBC query timeout (1s): " + e.getMessage(), | ||
| isQueryTimedOutAfterOneSecondMessage(e.getMessage())); | ||
| assertFalse("Message should not claim 0 seconds: " + e.getMessage(), |
There was a problem hiding this comment.
Considering the previous assertion, is that assertion possible at all?
There was a problem hiding this comment.
(assertFalse(… "after 0 seconds") after the positive check in testQueryTimeout — “is that even possible?”)
Changes
Removed assertFalse(..., e.getMessage().contains("after 0 seconds")) from testQueryTimeout.
| + " t2 on t1.under_col = t2.under_col"); | ||
| fail("Expecting SQLTimeoutException"); | ||
| } catch (SQLTimeoutException e) { | ||
| assertNotNull(e); |
There was a problem hiding this comment.
Is it possible having an exception with null value in a catch block?
There was a problem hiding this comment.
(assertNotNull(e) in catch (SQLTimeoutException e) — can e be null?)
Changes
Removed assertNotNull(e) from both SQLTimeoutException catch blocks in these tests.
| assertNotNull(e); | ||
| assertTrue("Message should include session timeout (1s): " + e.getMessage(), | ||
| isQueryTimedOutAfterOneSecondMessage(e.getMessage())); | ||
| assertFalse("Message should not claim 0 seconds (HIVE-28265): " + e.getMessage(), |
There was a problem hiding this comment.
What is the benefits of putting the ticket number into assertions or comments?
There was a problem hiding this comment.
Dropped ticket id from assertion messages.
Kept HIVE-28265 and expected message behavior in testQueryTimeoutMessageUsesHiveConf Javadoc (and the constant / assertEquals text describes behavior without the ticket).
| assertNotNull(e); | ||
| assertTrue("Message should include session timeout (1s): " + e.getMessage(), | ||
| isQueryTimedOutAfterOneSecondMessage(e.getMessage())); | ||
| assertFalse("Message should not claim 0 seconds (HIVE-28265): " + e.getMessage(), |
There was a problem hiding this comment.
Considering the previous assertion, is that assertion possible at all?
There was a problem hiding this comment.
Removed assertFalse(..., "after 0 seconds") from testQueryTimeoutMessageUsesHiveConf.
| * Sentinel: no {@code SET hive.query.timeout.seconds} has been observed on this connection yet. | ||
| */ | ||
| static final long SESSION_QUERY_TIMEOUT_NOT_TRACKED = -1L; | ||
| private final AtomicLong sessionQueryTimeoutSeconds = new AtomicLong(SESSION_QUERY_TIMEOUT_NOT_TRACKED); |
There was a problem hiding this comment.
Thinking out loud: I wonder if a connection can have concurrency issue: I mean, you can have multiple individual connections to Hive, but inside a connection itself, can we have multiple hive statements in parallel?
I have no such use case in my mind, but let me ping Ayush about this question.
@ayushtkn , what do you think?
There was a problem hiding this comment.
a single JDBC Connection can be shared across multiple threads, and it is entirely possible to have multiple HiveStatement objects executing concurrently on the same connection (which maps to a single session on the HS2 side).
via Beeline or so maybe not but In Hive Server 2 (HS2), a single JDBC Connection corresponds to a single HS2 Session. You can absolutely execute multiple queries concurrently within the same session by spawning multiple threads on the client side, each using a different HiveStatement created from that single HiveConnection.
| * Records the effective {@code hive.query.timeout.seconds} (in seconds) after a successful | ||
| * {@code SET hive.query.timeout.seconds=...} on this connection. Used for JDBC timeout messages. | ||
| */ | ||
| void recordSessionQueryTimeoutFromSet(long seconds) { |
There was a problem hiding this comment.
Can we keep the getter..setter naming pattern?
There was a problem hiding this comment.
recordSessionQueryTimeoutFromSet(long) → setSessionQueryTimeoutSeconds(long)
getSessionQueryTimeoutSecondsTracked() → getSessionQueryTimeoutSeconds()
HiveStatement updated to call the new names.
- TestJdbcDriver2: assert exact 'Query timed out after 1 seconds'; drop redundant assertNotNull/assertFalse; keep HIVE-28265 context in Javadoc not assert text - HiveConnection: AtomicLong Javadoc for concurrent statements; rename to setSessionQueryTimeoutSeconds / getSessionQueryTimeoutSeconds - HiveStatement: update call sites Made-with: Cursor
| log.info("Query timed out after: {} seconds. Cancelling the execution now: {}", queryTimeout, queryId); | ||
| setOperationException(new HiveSQLException( | ||
| "Query timed out after " + queryTimeout + " seconds", | ||
| "HYT00", |
There was a problem hiding this comment.
Could you please share some reference to this error code? The closest thing that I have found is a login timeout for SQL Server: https://learn.microsoft.com/en-us/answers/questions/1348638/sql-state-hyt00-sql-error-code-0-login-timeout-exp
There was a problem hiding this comment.
HYT00 is not a SQL Server–only code. It comes from the ODBC SQLSTATE list. In ODBC 3.x, HYT00 means “Timeout expired” (generic timeout, not specifically “login timeout”). SQL Server’s docs often show up in web search because they map ODBC states, but the same state is used for query / statement timeouts and other “timeout expired” situations when products follow ODBC-style SQLSTATEs.
References:
==> Microsoft’s ODBC appendix lists ODBC error / SQLSTATE codes; HYT00 is documented there in the “Timeout expired” family (see Appendix A: ODBC Error Codes — search the page for HYT00).
==> Broader context: HY = CLI/driver advisory category; T00 = timeout expired in that ODBC mapping (ODBC 2.x S1T00 was aligned to HYT00 in ODBC 3.x per SQLSTATE Mappings).
So Hive reusing HYT00 for query timeout is consistent with “timeout expired” in ODBC/JDBC-style SQLSTATE usage, not an odd SQL Server login-only choice. The Learn article they found is one example of HYT00, not the definition of the code.
| } | ||
| Matcher m = SET_HIVE_QUERY_TIMEOUT_SECONDS.matcher(sql); | ||
| Long lastSec = null; | ||
| while (m.find()) { |
There was a problem hiding this comment.
I found this PR interesting.
Unfortunately, I have no time to finish this review as I go for a long vacation.
But this part made me suspicious as I'm pretty sure we usually don't read Hive this way.
So, I read the Jira ticket itself: https://issues.apache.org/jira/browse/HIVE-28265
My fault, I should start with this one.
Actually, the ticket says the feature itself is already working but we get a wrong error message.
Just thinking out loud:
As I see HiveStatement doesn't contain any reference to Hive Configuration. Creating a hiveConf object is not a top of my mind but I affraid with this way you ignore the actual HiveConf loaded in the Hive Server session. I'm sad that I have no time to debug it out but for me, it looks suspicious.
I bet Hive already has it's method to read the SET ... commands out. As you can see, we have no such (or similar) parsing logic to read the Hive settings but still, if you run a set command, you can easily read the value from HiveConf. It would worth doing a debug session and figuring out how Hive exactly handles session level configurations.
Based on that I would say maybe there is a place where HiveStatement.setQueryTimeout should be called but it is not.
Anyway, good luck with the PR. If you have still have open questions at the end of the next week, I would be happy to help and learn this part of the code.
There was a problem hiding this comment.
thank you @InvisibleProgrammer This was reproducable.
Could you please let me know any setting or runtime configs, if you feel can fix the issue?
Could you also please let me know, how to refactor? As I am new to the community
I'm sad that I have no time to debug it out but for me, it looks suspicious.
HiveSQLException appends '; Query ID: ...' to getMessage(). When the client passes through the server timeout text (non-broken path), SQLTimeoutException included that suffix and TestJdbcDriver2 exact assertions failed on CI. Strip the same trailer the server uses before returning the message. Made-with: Cursor
|
There was a problem hiding this comment.
@ayushtkn @InvisibleProgrammer could you please review
4 participants
What changes were proposed in this pull request?
hive.query.timeout.seconds is enforced correctly, but Beeline/JDBC reported Query timed out after 0 seconds when Statement.setQueryTimeout was not used.
This PR:
SQLOperation (HiveServer2)
Before cancel(OperationState.TIMEDOUT), set a HiveSQLException whose message is Query timed out after seconds, using the effective operation timeout ( is the same value used to schedule the cancel). GetOperationStatus then exposes the right text via operationException.
For async execution, do not call setOperationException from the background thread if the operation is already TIMEDOUT, so the timeout message is not overwritten.
HiveStatement (JDBC)
On TIMEDOUT_STATE, prefer the server errorMessage. If it is missing or clearly wrong (contains after 0 seconds), build the client message from Statement query timeout or from the last SET hive.query.timeout.seconds=... value tracked on HiveConnection. SET is detected with a regex find() so assignments can appear inside a longer script (last match wins).
HiveConnection
Stores the last parsed hive.query.timeout.seconds from a successful SET for use in the timeout message when needed.
Tests
==> testQueryTimeoutMessageUsesHiveConf: session SET hive.query.timeout.seconds=1s, no setQueryTimeout, slow query via existing SleepMsUDF — expects SQLTimeoutException and message not claiming after 0 seconds, and containing 1.
==> testQueryTimeout: same checks for the existing setQueryTimeout(1) path.
-->
Why are the changes needed?
Does this PR introduce any user-facing change?
How was this patch tested?