Issue 6056 - WebUI supports only instances with BDB #6299

jchapma · 2024-08-14T13:03:23Z

Description:
BDB specific attributes are not supported by the web console, resulting in an "Ooops" when the Database tab is selected.

Fix Description:
Add a new global Database configuration element for MDB layout. Add support to detect what DB engine is configured, and display the correct layout on rendering.

Relates: #6065

Reviewed by:

mreynolds389 · 2024-08-14T13:58:09Z

src/cockpit/389-console/src/lib/database/databaseConfig.jsx

+                                <div className="ds-left-indent-md">
+                                    <Grid
+                                        title={_("Database maximum size in bytes. The practical maximum size of an LMDB database is limited by the system’s addressable memory (nsslapd-mdb-max-size).")}
+                                        className="ds-margin-top"


I noticed several fields are very close to bottom of the tabs, I think it would look a little nicer if you used this class instead: ds-margin-top-xlg

Do this for all the grids below so they are consistent, thanks!

I agree, it looks way better

vashirov

Few small issues. For the rest I need to try the PR locally.

vashirov · 2024-08-14T14:45:21Z

src/cockpit/389-console/src/lib/database/databaseConfig.jsx

+                                        className="ds-margin-top"
+                                    >
+                                        <GridItem className="ds-label" span={4}>
+                                            {_("Database Max DBS")}


Suggested change

{_("Database Max DBS")}

{_("Database Max DBs")}

vashirov · 2024-08-14T14:55:12Z

src/cockpit/389-console/src/lib/database/databaseConfig.jsx

+                            <Tab eventKey={0} title={<TabTitleText>{_("Database Size")}</TabTitleText>}>
+                                <div className="ds-left-indent-md">
+                                    <Grid
+                                        title={_("Database maximum size in bytes. The practical maximum size of an LMDB database is limited by the system’s addressable memory (nsslapd-mdb-max-size).")}


Maximum size should be a multiple of the page size, which can vary (4K, 16K, 64K on different architectures). And as a user, I'd prefer to enter the size in megabytes, than calculate the right size.

I like that idea, let me look at how to determine the system page size and make the max size a multiple of it

1MB contains 256 4K pages or 64 16K pages or 16 64K pages. So if we change the maximum size to use megabytes, it'd cover all cases.

vashirov · 2024-08-14T14:55:51Z

src/cockpit/389-console/src/lib/database/databaseConfig.jsx

+                            <Tab eventKey={5} title={<TabTitleText>{_("Advanced Settings")}</TabTitleText>}>
+                                <div className="ds-left-indent-md">
+                                    <Grid
+                                        title={_("Location for database memory mapped files.  You must specify a subdirectory of a tempfs type filesystem (nsslapd-db-home-directory).")}


Suggested change

title={_("Location for database memory mapped files. You must specify a subdirectory of a tempfs type filesystem (nsslapd-db-home-directory).")}

title={_("Location for database memory mapped files. You must specify a subdirectory of a tempfs type filesystem (nsslapd-db-home-directory).")}

progier389 · 2024-08-20T13:16:04Z

Not sure we want to configure nsslapd-import-cachesize in lmdb case (Although it is still used in some computation
I think it is dead code, neither the entries nor the index are cached in lmdb import )
so IMHO it will be better to simply remove that tab

mreynolds389 · 2024-08-20T13:22:59Z

Not sure we want to configure nsslapd-import-cachesize in lmdb case (Although it is still used in some computation I think it is dead code, neither the entries nor the index are cached in lmdb import ) so IMHO it will be better to simply remove that tab

I think the import cache is used for other things that are not DB related (regardless of BDB/MDB) - so it still might be needed. Just proceed with caution :-)

droideck · 2024-08-16T02:30:15Z

src/cockpit/389-console/src/database.jsx

@@ -88,6 +90,7 @@ export class Database extends React.Component {
            chainingConfig: {},
            chainingUpdated: 0,
            chainingActiveKey: 0,
+            backendImplement: "mdb",


Why not to use BE_IMPL_MDB const here?

droideck · 2024-08-20T19:08:08Z

src/cockpit/389-console/src/lib/database/databaseConfig.jsx

+                                                value={this.state.dbhomedir}
+                                                type="text"
+                                                id="dbhomedir"
+                                                aria-describedby="dbhomedir"


For some reason, when I change dbhome directory, the button still stays grey. And if I change something else and then save, dbhome dir still resets to the default - /var/lib/dirsrv/slapd-inst/db

And it happens only for LMDB. BDB is fine

Ok, the button change is a copy and paste error.

Regarding the dbhome setting, when I try changing the db home directory with bdb, an unwilling to perform message is returned, in the error logs it mentions this config cannot be changed when the server is running. IIUC then do we need a config option for changing the db home in the UI at all?

WDYT

droideck · 2024-08-20T19:14:26Z

src/cockpit/389-console/src/lib/database/databaseConfig.jsx

+    handleTimeChange(value) {
+        this.setState({
+            compacttime: value,
+        }, () => { this.validateSaveBtn() });
+    }


I think we don't need this in LMDB part

You are correct, removing this

jchapma · 2024-08-21T22:57:35Z

Not sure we want to configure nsslapd-import-cachesize in lmdb case (Although it is still used in some computation I think it is dead code, neither the entries nor the index are cached in lmdb import ) so IMHO it will be better to simply remove that tab

I think the import cache is used for other things that are not DB related (regardless of BDB/MDB) - so it still might be needed. Just proceed with caution :-)

Let me look into this, cautiously...

progier389 · 2024-08-22T14:12:03Z

Not sure we want to configure nsslapd-import-cachesize in lmdb case (Although it is still used in some computation I think it is dead code, neither the entries nor the index are cached in lmdb import ) so IMHO it will be better to simply remove that tab

I think the import cache is used for other things that are not DB related (regardless of BDB/MDB) - so it still might be needed. Just proceed with caution :-)

Apparently no, according to 'grep -ri import.cache' it is only used by bdb:
Once the config handling removed, you found reference in db-bdb code (and only one in db-mdb but only to set job_index_buffer_size which is itself no more used)

And I do not think that removing a now obscure parameter from cockpit is really dangerous (people can still view/change it through dsconf anyway), I am afraid that if we left it, some people may ask questions about it and we will be in trouble to answer ...

jchapma · 2024-08-22T14:15:27Z

Not sure we want to configure nsslapd-import-cachesize in lmdb case (Although it is still used in some computation I think it is dead code, neither the entries nor the index are cached in lmdb import ) so IMHO it will be better to simply remove that tab

I think the import cache is used for other things that are not DB related (regardless of BDB/MDB) - so it still might be needed. Just proceed with caution :-)

Apparently no, according to 'grep -ri import.cache' it is only used by bdb: Once the config handling removed, you found reference in db-bdb code (and only one in db-mdb but only to set job_index_buffer_size which is itself no more used)

And I do not think that removing a now obscure parameter from cockpit is really dangerous (people can still view/change it through dsconf anyway), I am afraid that if we left it, some people may ask questions about it and we will be in trouble to answer ...

Thanks for the clarification

mreynolds389 · 2024-08-22T15:57:40Z

Not sure we want to configure nsslapd-import-cachesize in lmdb case (Although it is still used in some computation I think it is dead code, neither the entries nor the index are cached in lmdb import ) so IMHO it will be better to simply remove that tab

I think the import cache is used for other things that are not DB related (regardless of BDB/MDB) - so it still might be needed. Just proceed with caution :-)

Apparently no, according to 'grep -ri import.cache' it is only used by bdb: Once the config handling removed, you found reference in db-bdb code (and only one in db-mdb but only to set job_index_buffer_size which is itself no more used)

Right, the indexing buffer. I knew it wasn't an internal DB cache, but forgot exactly how it was used. Ok if it's not used by MBM then it's safe to drop. Thanks!!

Description: BDB specific attributes are not supported by the web console, resulting in an "Ooops" when the Database tab is selected. Fix Description: Add a new global Database configuration element for MDB layout. Add support to detect what DB engine is configured, and display the correct layout on rendering. Relates: 389ds#6065 Reviewed by:

droideck

Looks good to me!
I think, @vashirov's ack is also needed, though. :)

vashirov

Database Maximum Size hint says the size is in bytes, but the form accepts megabytes now.

If Database Max Readers set to one of the numbers within 1-25 range, it gets set to 26. Not sure if it's possible to exclude this range from setting in the first place, @droideck do you know?

Database Max DBs has a minimum value of 36 that can be set, but if set it to anything within 36-68 range, it gets reset to 69. Should 69 be the minimum then? I don't know where it comes from though.

jchapma · 2024-09-12T21:47:58Z

Database Maximum Size hint says the size is in bytes, but the form accepts megabytes now.

If Database Max Readers set to one of the numbers within 1-25 range, it gets set to 26. Not sure if it's possible to exclude this range from setting in the first place, @droideck do you know?

Database Max DBs has a minimum value of 36 that can be set, but if set it to anything within 36-68 range, it gets reset to 69. Should 69 be the minimum then? I don't know where it comes from though.

I will correct the first issue.

I see the same behaviour when I run the associated commands at the CLI level:

sudo dsconf ldapi://%2fvar%2frun%2fslapd-inst01.socket backend config set --mdb-max-readers=25
sudo dsconf ldapi://%2fvar%2frun%2fslapd-inst01.socket backend config get | grep readers
nsslapd-mdb-max-readers: 26

And the same for max dbs:

sudo dsconf ldapi://%2fvar%2frun%2fslapd-inst01.socket backend config set --mdb-max-dbs=36
sudo dsconf ldapi://%2fvar%2frun%2fslapd-inst01.socket backend config get | grep "max-dbs"
nsslapd-mdb-max-dbs: 131

I suspect the issue is not at the UI level as it gets/sets values via dsconf over ldapi. Let me have a look into it.

droideck · 2024-09-17T00:37:38Z

Database Maximum Size hint says the size is in bytes, but the form accepts megabytes now.

If Database Max Readers set to one of the numbers within 1-25 range, it gets set to 26. Not sure if it's possible to exclude this range from setting in the first place, @droideck do you know?

Database Max DBs has a minimum value of 36 that can be set, but if set it to anything within 36-68 range, it gets reset to 69. Should 69 be the minimum then? I don't know where it comes from though.

Technically, yes, <NumberInput> allows to set min and max values. So, we can probably align them with the CLI output.

vashirov · 2024-09-17T09:32:16Z

@droideck, does it allow to have a sparse range? I.e. allow to set 0 and then 26-512? If not, I'm fine with the current behaviour.

droideck · 2024-09-17T15:18:06Z

@droideck, does it allow to have a sparse range? I.e. allow to set 0 and then 26-512? If not, I'm fine with the current behaviour.

For that, I see two options:

We can just add a checkbox on the side that will force 0 (and it will disable the field when checked) which means this value is computed by the server.
We can modify the <NumberInput> callbacks to handle the sparse range. Currently, we call it like this: onChange={(e) => { this.onConfigChange(e, "mdbmaxreaders", 0, 200) }} And we can change its code any way we like.
Same with onMinus={() => { this.onMinusConfig("mdbmaxreaders") }} and onPlus={() => { this.onPlusConfig("mdbmaxreaders") }}

jchapma · 2024-09-18T22:57:04Z

I didn't realise there was a sparse range of values allowed for these mdb configs, I guess I could have fuzzed the CLI instead of pointing the finger at the layer underneath the UI, sorry @progier389

I would prefer @droideck option 2, where we handle the logic in changeConfig() , onMinus() and onPlus(), it feels cleaner to me.
Thanks

mreynolds389 reviewed Aug 14, 2024

View reviewed changes

vashirov reviewed Aug 14, 2024

View reviewed changes

droideck force-pushed the ui-lmdb branch from b038a59 to ab51bf2 Compare August 20, 2024 15:14

droideck requested changes Aug 20, 2024

View reviewed changes

jchapma added 2 commits August 29, 2024 09:35

rework

e4b93d7

droideck approved these changes Sep 10, 2024

View reviewed changes

vashirov reviewed Sep 11, 2024

View reviewed changes

sparse range updates

9583145

jchapma force-pushed the ui-lmdb branch from 96a4a5f to 9583145 Compare September 19, 2024 23:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue 6056 - WebUI supports only instances with BDB #6299

Issue 6056 - WebUI supports only instances with BDB #6299

jchapma commented Aug 14, 2024

mreynolds389 Aug 14, 2024

jchapma Aug 21, 2024

vashirov left a comment

vashirov Aug 14, 2024

vashirov Aug 14, 2024

jchapma Aug 21, 2024

vashirov Aug 22, 2024

vashirov Aug 14, 2024

progier389 commented Aug 20, 2024

mreynolds389 commented Aug 20, 2024

droideck Aug 16, 2024

droideck Aug 20, 2024

jchapma Aug 21, 2024

droideck Aug 20, 2024

jchapma Aug 21, 2024

jchapma commented Aug 21, 2024

progier389 commented Aug 22, 2024

jchapma commented Aug 22, 2024

mreynolds389 commented Aug 22, 2024

droideck left a comment

vashirov left a comment

jchapma commented Sep 12, 2024 •

edited

Loading

droideck commented Sep 17, 2024

vashirov commented Sep 17, 2024

droideck commented Sep 17, 2024

jchapma commented Sep 18, 2024 •

edited

Loading

	title={_("Location for database memory mapped files. You must specify a subdirectory of a tempfs type filesystem (nsslapd-db-home-directory).")}
	title={_("Location for database memory mapped files. You must specify a subdirectory of a tempfs type filesystem (nsslapd-db-home-directory).")}

Issue 6056 - WebUI supports only instances with BDB #6299

Are you sure you want to change the base?

Issue 6056 - WebUI supports only instances with BDB #6299

Conversation

jchapma commented Aug 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vashirov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

progier389 commented Aug 20, 2024

mreynolds389 commented Aug 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jchapma commented Aug 21, 2024

progier389 commented Aug 22, 2024

jchapma commented Aug 22, 2024

mreynolds389 commented Aug 22, 2024

droideck left a comment

Choose a reason for hiding this comment

vashirov left a comment

Choose a reason for hiding this comment

jchapma commented Sep 12, 2024 • edited Loading

droideck commented Sep 17, 2024

vashirov commented Sep 17, 2024

droideck commented Sep 17, 2024

jchapma commented Sep 18, 2024 • edited Loading

jchapma commented Sep 12, 2024 •

edited

Loading

jchapma commented Sep 18, 2024 •

edited

Loading