Re-implement SET STATEMENT ... FOR ... using 8.0 hint syntax

Description

In 8.0 we are keeping the per-statement variable setting feature but dropping our/MariaDB syntax in favor of upstream statement hint syntax, which, in upstream, only works for a subset of variables on SELECTs. This task is to use this syntax for all statements, virtually all variables.

This can be done in parallel with main 8.0 porting, on a clean upstream 8.0.11 branch as base.

 

High Level Description

SET STATEMENT FOR provides a way to set multiple session variables for a single statement. The goal of the task is replace percona’s implementation(syntax) but provide same functionality using MySQL 8.0 SET_VAR().

High Level Design

  1. SET_VAR hints works will all DML statements. ie. INSERT, UPDATE, DELETE

  2. SET_VAR hints works will all SELECTs

  3. SET_VAR works with PREPARE STATEMENT

  4. SET_VAR hints are not replicated whereas SET STATEMENT FOR variables are replicated. In 8.0, Percona server will use upstream behaviour. i.e. not to replicate variables set using SET_VAR hints

  5. SET_VAR works with stored procedure. SET_VAR hints  can be at statement level inside procedure or an hint to the CALL statement. The hint cannot be written at the definition of the complete stored procedure. Example:

                   DELIMITER |;

                  SET STATEMENT myisam_sort_buffer_size=400000,

                                          myisam_repair_threads=2,

                                           sort_buffer_size=200000,

                                            binlog_format=row,

                                           keep_files_on_create=OFF,

                                          max_join_size=4444440000000 FOR

                                          CREATE PROCEDURE p1() BEGIN

                                          SELECT @@myisam_sort_buffer_size,

                                                 @@myisam_repair_threads,

                                                 @@sort_buffer_size,

                                                 @@binlog_format,

                                                 @@keep_files_on_create,

                                                 @@max_join_size;

                                           END|

                  DELIMITER ;|

                  We cannot have  /*+ SET_VAR(sort_buffer_size=20000) */ CREATE PROCEDURE

                 p1() .   

                 SET_VAR hints should be written at STATEMENT. (after SELECT/UPDATE/DELETE..

                 ) or  CALL /*+SET_VAR() */ p1(). The hints at CALL statement will be applicable to all

                 statements inside stored procedure.

                The hints at statement within stored procedure will have higher preference compared to

                hints at CALL statement.

      6. SET_VAR hints do not work in subqueries

      7. Recursive SET STATEMENT FOR  are possible but recursive SET_VAR are not.

          Percona Server will use upstream behaviour. For example, the equivalent of the below

           statement using SET_VAR will not be possible.

                    SET STATEMENT myisam_sort_buffer_size = 700000, sort_buffer_size = 3000000

                    FOR SET STATEMENT myisam_sort_buffer_size=200000

                      FOR SELECT @@myisam_sort_buffer_size, @@sort_buffer_size;

     8.  Hints are not supported in view (CREATE or ALTER VIEW)

     9.  Setting from global values is not allowed

           Example: SELECT /*+ SET_VAR(sort_buffer_size=@@global.sort_buffer_size) */ * FROM

           t1;

Percona Server changes (extends SET_VAR to):

  1. OPTIMIZE TABLE statement

  2.  MyISAM session variables

  3.  Plugin or Storage Engine variables

  4. InnoDB Session variables. innodb_lockwait_timeout and innodb_tmpdir

  5. ALTER TABLE. Works with variables: innodb_tmpdir , innodb_ft_user_stopword_table

  6. CALL stored_proc() statement

  7. ANALYZE TABLE statement. Works with session variables "histogram_generation_max_mem_size" and "myisam_stats_method"

  8. CHECK TABLE statement. Works with  session variable "myisam_stats_method"

  9. LOAD INDEX statement(used for MyiSAM). Works with session variable "preload_buffer_size"

  10. CREATE TABLE statement. Works with session variable  "innodb_ft_user_stopword_table"

List of variables that are extended to use with SET_VAR

  1. innodb_lock_wait_timeout

  2. innodb_tmpdir

  3. innodb_ft_user_stopword_table

  4. block_encryption_mode

  5. histogram_generation_max_mem_size

  6. myisam_sort_buffer_size

  7. myisam_repair_threads

  8. myisam_stats_method

  9. preload_buffer_size(used by MyISAM only)

Environment

None

has to be done before

Smart Checklist

Activity

Show:

Satya Bodapati September 10, 2018 at 11:34 AM

Done!

Laurynas Biveinis September 10, 2018 at 9:21 AM

please update the JIRA status after the trunk merge

Satya Bodapati August 24, 2018 at 8:09 AM
Edited

the set_var grammar is in separate file and we cannot use the existing rules defined for "variable". So duplicated the rules in set_var grammar file. Still we get the below mentioned errors. After checking with Yura, we concluded that it is not straightforward to implement this. The parser for set_var is limited and not even YYTHD is available with SET_VAR grammar.

Conclusion: We take this up as separate task after the initial work is pushed

Patch that was tried for enabling SET_VAR(var_name=@@global.var_name)

diff --git a/sql/sql_hints.yy b/sql/sql_hints.yy
index df80fddacef..2d119d0f547 100644
— a/sql/sql_hints.yy
+++ b/sql/sql_hints.yy
@@ -89,6 +89,12 @@ static bool parse_int(longlong *to, const char *from, size_t from_length)
 %token INDEX_MERGE_HINT
 %token NO_INDEX_MERGE_HINT
 %token SET_VAR_HINT
+%token HINT_IDENT
+%token HINT_IDENT_QUOTED
+%token HINT_TEXT_STRING
+%token<keyword> HINT_GLOBAL_SYM
+%token<keyword> HINT_LOCAL_SYM
+%token<keyword> HINT_SESSION_SYM
 
 /* Other tokens */
 
@@ -150,11 +156,26 @@ static bool parse_int(longlong *to, const char *from, size_t from_length)
   set_var_num_item
   set_var_string_item
   set_var_arg
+  variable
+  variable_aux
 
 %type <ulong_num>
   semijoin_strategy semijoin_strategies
   subquery_strategy
 
+%type <lex_str>
+  HINT_IDENT
+  HINT_IDENT_QUOTED
+  hint_ident_or_text
+  hint_TEXT_STRING_sys
+  HINT_TEXT_STRING
+  hint_opt_component
+  hint_IDENT_sys
+  hint_ident
+
+%type <var_type>
+  hint_opt_var_ident_type
+
 %%
 
 
@@ -604,8 +625,88 @@ set_var_string_item:
           if ($$ == NULL)
             YYABORT; // OOM
         }
+
+variable:
+          '@' variable_aux { $$= $2; }
+        ;
+
+variable_aux:
+        '@' hint_opt_var_ident_type hint_ident_or_text hint_opt_component
+          {
+            $$= NEW_PTN PTI_variable_aux_3d(@$, $2, $3, @3, $4);
+          }
+        ;
+
+hint_opt_var_ident_type:
+          /* empty */     { $$=OPT_DEFAULT; }
+        | HINT_GLOBAL_SYM '.'  { $$=OPT_GLOBAL; }
+        | HINT_LOCAL_SYM '.'   { $$=OPT_SESSION; }
+        | HINT_SESSION_SYM '.' { $$=OPT_SESSION; }
+        ;
+
+hint_opt_component:
+          /* empty */    { $$= null_lex_str; }
+        | '.' hint_ident      { $$= $2; }
+        ;
+
+hint_ident_or_text:
+          hint_ident           { $$=$1;}
+        | hint_TEXT_STRING_sys { $$=$1;}
+        ;
+
+hint_ident:
+          hint_IDENT_sys    { $$=$1; }
+        ;
+
+hint_IDENT_sys:
+          HINT_IDENT { $$= $1; }
+        | HINT_IDENT_QUOTED
+          {
+            THD *thd= YYTHD;
+
+            if (thd->charset_is_system_charset)
+            {
+              const CHARSET_INFO *cs= system_charset_info;
+              int dummy_error;
+              size_t wlen= cs->cset->well_formed_len(cs, $1.str,
                                                     $1.str$1.length,
+                                                     $1.length, &dummy_error);
+              if (wlen < $1.length)
+              {
+                ErrConvString err($1.str, $1.length, &my_charset_bin);
+                my_error(ER_INVALID_CHARACTER_STRING, MYF(0),
+                         cs->csname, err.ptr());
+                MYSQL_YYABORT;
+              }
+              $$= $1;
+            }
+            else
+            {
+              if (thd->convert_string(&$$, system_charset_info,
+                                  $1.str, $1.length, thd->charset()))
+                MYSQL_YYABORT;
+            }
+          }
+        ;
+
+hint_TEXT_STRING_sys:
+          HINT_TEXT_STRING
+          {
+            THD *thd= YYTHD;
+
+            if (thd->charset_is_system_charset)
+              $$= $1;
+            else
+            {
+              if (thd->convert_string(&$$, system_charset_info,
+                                  $1.str, $1.length, thd->charset()))
+                MYSQL_YYABORT;
+            }
+          }
+        ;
 
 set_var_arg:
     set_var_string_item
     | set_var_num_item
+    | variable
     ;

===========

Errors seen:

/home/satya/WORK/mysql-8.0.11-set-statement/bld/sql/sql_hints.yy.cc: In function ‘int HINT_PARSER_parse(THD*, Hint_scanner*, PT_hint_list**)’:
/home/satya/WORK/mysql-8.0.11-set-statement/bld/sql/sql_hints.yy.cc:1481:48: error: cannot convert ‘YYLTYPE*’ to ‘Hint_scanner*’ for argument ‘2’ to ‘int HINT_PARSER_lex(YYSTYPE*, Hint_scanner*)’
      yychar = yylex (&yylval, &yylloc, scanner);
                                               ^
/home/satya/WORK/mysql-8.0.11-set-statement/sql/sql_hints.yy:636:35: error: expected type-specifier before ‘PTI_variable_aux_3d’
            $$= NEW_PTN PTI_variable_aux_3d(@$, $2, $3, @3, $4);
                                  ^~~~~~~~~~~~~~~~~~~
/home/satya/WORK/mysql-8.0.11-set-statement/sql/sql_hints.yy:665:23: error: ‘YYTHD’ was not declared in this scope
            THD *thd= YYTHD;

 

 

 

 

Satya Bodapati August 15, 2018 at 3:30 PM

1) CALL /*+ SET_VAR(sort_buffer_size=10000) stored_proce_name() is working now! So we now have equivalent functionality as SET STATEMENT FOR.

What I found is same thd is used for for multiple mysql_execute_command() and lex and some stuff is reset after each statement.

Now onto make this work: SET_VAR(sort_buffer_size=default) AND SET_VAR(sort_buffer_size=@@global.sort_buffer_size)

Satya Bodapati August 14, 2018 at 10:40 AM

1) it is not wrong syntax but the SET_VAR hint is ignored. I will check more on SET_VAR architecture. AFAIK, each mysql_execute_command() will read hint value and restore at the end of execution.  So from stored procedures, each statemnt would be one mysql_execute_command.

May be it will be possible to have hints with the CALL statement. Will need some more time to understand feasibility

2) I guess it is not about a setting that will make CREATE VIEW fail. A invalid SET_VAR always creates warning (never makes statement fail). Like I said above, it may be about how the parsed value is stored, applied and restored.

3) Right, I will check on how to make this work. It seems useful.

Also working on extending ALTER to take hints (specially for innodb_tmpdir).

 

Done

Details

Assignee

Reporter

Time tracking

1w 1h 43m logged

Fix versions

Priority

Smart Checklist

Created April 20, 2018 at 9:39 AM
Updated August 30, 2021 at 9:36 AM
Resolved September 10, 2018 at 11:34 AM