Set check_table_dependencies to 0 to avoid HAVE_DEPENDENT_OBJECTS errors in restore

[Alex-Burmak]

$ ch-backup --port 29443 --config /etc/yandex/ch-backup/ch-backup-schema-copy.conf restore-access-control 9b604b87-8feb-4f50-ad57-bacbbd7271ad; /usr/bin/ch-backup --port 29443 --config /etc/yandex/ch-backup/ch-backup-schema-copy.conf restore 9b604b87-8feb-4f50-ad57-bacbbd7271ad --schema-only
Traceback (most recent call last):
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/ch_backup/clickhouse/client.py", line 64, in query
    response.raise_for_status()
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://sas-t1am73537l8hmc1k.db.yandex.net:29443/?allow_experimental_database_materialized_postgresql=1&allow_experimental_database_replicated=1&allow_experimental_funnel_functions=1&allow_experimental_live_view=1&allow_suspicious_codecs=1&allow_suspicious_low_cardinality_types=1&allow_experimental_database_materialized_mysql=1&allow_experimental_nlp_functions=1&allow_experimental_window_view=1&allow_experimental_object_type=1&allow_experimental_hash_functions=1&allow_deprecated_database_ordinary=1&allow_deprecated_syntax_for_merge_tree=1&kafka_disable_num_consumers_limit=1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/ch-backup", line 11, in <module>
    load_entry_point('ch-backup==2.469.8602709', 'console_scripts', 'ch-backup')()
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/ch_backup/cli.py", line 161, in wrapper
    result = ctx.invoke(f, ctx, ctx.obj["backup"], *args, **kwargs)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/ch_backup/cli.py", line 709, in restore_command
    keep_going=keep_going,
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/ch_backup/ch_backup.py", line 299, in restore
    keep_going=keep_going,
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/ch_backup/ch_backup.py", line 552, in _restore
    keep_going=keep_going,
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/ch_backup/logic/table.py", line 232, in restore
    context, databases, tables_to_restore
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/ch_backup/logic/table.py", line 502, in _preprocess_tables_to_restore
    context.ch_ctl.drop_dictionary_if_exists(table)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/ch_backup/clickhouse/control.py", line 527, in drop_dictionary_if_exists
    db_name=escape(table.database), table_name=escape(table.name)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/tenacity/__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/tenacity/__init__.py", line 314, in iter
    return fut.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
  File "/opt/yandex/ch-backup/lib/python3.6/site-packages/ch_backup/clickhouse/client.py", line 66, in query
    raise ClickhouseError(e.response.text.strip()) from e
ch_backup.clickhouse.client.ClickhouseError: Code: 630. DB::Exception: Cannot drop or rename dict.auto_tech_support_user_id_d, because some tables depend on it: prod_dm.auto_tech_support_user_id. (HAVE_DEPENDENT_OBJECTS) (version 22.8.20.11 (official build))

[aalexfvk]

Does it make sense to add command line option --ignore-dependencies for restore command or there are no cases when it would be useful ?

[Alex-Burmak]

Does it make sense to add command line option --ignore-dependencies for restore command or there are no cases when it would be useful ?

Not sure that it would be useful. Right now we use DROP TABLE inside restore command as a part of retry logic.