Enabling/disabling subscribed lists

hartacus · April 11, 2025, 6:46am

The Subscribed Lists GUI has switches to enable and disable lists, but these aren't working properly for me.

Attempting to disable an enabled list doesn't seem to disable it becaue the switch automatically reverts to enabled. But it also creates a disabled copy of the same list, and the switch works on the copy. Both seem to operate independently for gravity sync so it's difficult to actually disable a list.

This behaviour seemed to stop after I sorted by several different columns one one of my devices, but I haven't been able to get it to stop on the other device.

Tricorder token: https://tricorder.pi-hole.net/oj0XsIrj/

yubiuser · April 11, 2025, 7:52am

You got database errors.

  2025-04-11 15:41:24.188 AEST [20019M] ERROR: SQLite3: no such table: main.antigravity in "SELECT adlist_id from vw_antigravity WHERE domain = ? AND group_id IN (0,3);" (1)
   2025-04-11 15:41:24.189 AEST [20019M] ERROR: domain_in_list("web.facebook.com", 0xb6dae958, antigravity): Failed to perform step: SQL logic error
   2025-04-11 15:41:24.189 AEST [20019M] ERROR: SQLite3: no such table: main.gravity in "SELECT adlist_id from vw_gravity WHERE domain = ? AND group_id IN (0,3);" (1)
   2025-04-11 15:41:24.190 AEST [20019M] ERROR: domain_in_list("web.facebook.com", 0xb6f39938, gravity): Failed to perform step: SQL logic error
   2025-04-11 15:41:24.193 AEST [20019M] ERROR: SQLite3: no such table: main.antigravity in "SELECT adlist_id from vw_antigravity WHERE domain = ? AND group_id IN (0,3);" (1)

Try to make a teleporter backup, move /etc/pihole/gravity.db to a backup location, run gravity, restore the teleporter backup

hartacus · April 22, 2025, 5:24am

Hi, thanks for your reply. I've tried those steps, and they haven't changed the issue. Here's another tricorder log after following those steps: https://tricorder.pi-hole.net/5RxUV5sC/

hartacus · April 22, 2025, 6:34am

I noticed something else unusual about this: there is a slash dropped from the list URL of the duplicate. I think this means it isn't correctly recognising the record that is already there.

I enabled API debugging and replicated the issue, and you can see that the slash is dropped from the API call, so there would be no address conflict to prompt the SQL to update rather than insert.

2025-04-22 16:01:39.683 AEST [20019/T31383] DEBUG_API: Requested API URI: ::1 -> PUT /api/lists/https:/v.firebog.net/hosts/Easylist.txt ? type=block (Content-Type application/json; charset=utf-8)
2025-04-22 16:01:39.684 AEST [20019/T31383] DEBUG_API: Received payload with size: 66
2025-04-22 16:01:39.685 AEST [20019/T31383] DEBUG_API: Read sid="[redacted]" from cookie
2025-04-22 16:01:39.686 AEST [20019/T31383] DEBUG_API: Recognized known user: user_id 0, valid_until: 2025-04-22 16:31:39 AEST, remote_addr ::1 (::1 at login)
2025-04-22 16:01:39.687 AEST [20019/T31383] DEBUG_API: Processing PUT /api/lists/https:/v.firebog.net/hosts/Easylist.txt in /api/lists
2025-04-22 16:01:39.703 AEST [20019/T31383] DEBUG_API: SQL: INSERT INTO adlist (address,enabled,comment,type) VALUES (:item,:enabled,:comment,:type) ON CONFLICT(address,type) DO UPDATE SET enabled = :enabled, comment = :comment, type = :type;
2025-04-22 16:01:39.704 AEST [20019/T31383] DEBUG_API:      :item = "https:/v.firebog.net/hosts/Easylist.txt"
2025-04-22 16:01:39.705 AEST [20019/T31383] DEBUG_API:      :type = 0
2025-04-22 16:01:39.706 AEST [20019/T31383] DEBUG_API:      :comment = "EasyList"
2025-04-22 16:01:39.706 AEST [20019/T31383] DEBUG_API:      :enabled = 0
2025-04-22 16:01:39.708 AEST [20019/T31383] DEBUG_API: SQL: SELECT id FROM adlist WHERE address = :item AND type = :type
2025-04-22 16:01:39.709 AEST [20019/T31383] DEBUG_API:      :item = "https:/v.firebog.net/hosts/Easylist.txt"
2025-04-22 16:01:39.710 AEST [20019/T31383] DEBUG_API:      :type = "0"
2025-04-22 16:01:39.722 AEST [20019/T31383] DEBUG_API: SQL: DELETE FROM adlist_by_group WHERE adlist_id = :id;
2025-04-22 16:01:39.723 AEST [20019/T31383] DEBUG_API:      :id = "68"
2025-04-22 16:01:39.724 AEST [20019/T31383] DEBUG_API: groupscount = 1
2025-04-22 16:01:39.736 AEST [20019/T31383] DEBUG_API: INSERT: 101 -> (68,0)
2025-04-22 16:01:39.737 AEST [20019/T31383] DEBUG_API: SQL: INSERT INTO adlist_by_group (adlist_id,group_id) VALUES (:id,:gid);
2025-04-22 16:01:39.738 AEST [20019/T31383] DEBUG_API:      :id = "68"
2025-04-22 16:01:39.739 AEST [20019/T31383] DEBUG_API:      :gid = "0"
2025-04-22 16:01:39.741 AEST [20019/T31383] DEBUG_API: SQL: SELECT id,type,address,enabled,date_added,date_modified,comment,(SELECT GROUP_CONCAT(group_id) FROM adlist_by_group g WHERE g.adlist_id = a.id) AS group_ids,date_updated,number,invalid_domains,status,abp_entries FROM adlist a WHERE type = 0 AND address = :item;
2025-04-22 16:01:39.742 AEST [20019/T31383] DEBUG_API:      :item = "https:/v.firebog.net/hosts/Easylist.txt"
2025-04-22 16:01:39.743 AEST [20019/T31383] DEBUG_API:      :ids = "(null)"
2025-04-22 16:01:39.751 AEST [20019/T31383] DEBUG_API: Done
2025-04-22 16:01:39.769 AEST [20019/T30836] DEBUG_ANY: Closing gravity database

The slash is present on the UI for the original record, so I'm not sure how it's dropped. The original record is the one at the bottom in this image.

rdwebdesign · April 22, 2025, 6:43pm

I couldn't find a way to insert a list address and lose one slash from the URL. Looks like it was a typo.

Your debug log and the screenshot above show other lists with a missing slash.
They are the 3 most recent entries in your database:

All older entries (including the older https://v.firebog.net/hosts/Easylist.txt) are correctly showing 2 slashes.

hartacus · April 22, 2025, 10:51pm

I can confirm it's not a typo, because I'm not typing it.

Those entries are created without the slash when I try to disable an existing list that I had entered with the slash correctly. I only just noticed the missing slashes, but previously I hadn't noticed a difference, so I guess I had deleted the original entries for those lists without noticing what had happened. I also can't find a way to drop a slash while entering a list, but that's not what's happening here, I'm not entering new lists, I'm trying to disable existing lists.

To make it clear, the above API debugging information was returned when I clicked the enable/disable slider of an existing entry with both slashes in place. I did not type in any address, I only clicked the enable/disable slider.

I've fixed the entries with the missing slash (re-added the entries with correctly formatted URLs and removed the single-slash entries). I can confirm that the issue still occurs - attempting to disable a list results in a new disabled list added with one slash dropped from the URL and the original list still enabled.
Before:

After clicking Enabled slider:

hartacus · April 22, 2025, 11:44pm

I found the cause of this: it was nginx sanitising the double slash out of the URL for the API call.

(I was using nginx as the web server for v5, switched to nginx reverse proxy for v6, supports my particular HTTPS setup.)

Adding merge_slashes off; to the server context of nginx config turned off the sanitisation and allowed the API calls to go through normally, which allows the list management to work as expected.
Source: nginx: How to keep double slashes in urls - Stack Overflow

API call after applying this, note correctly formatted list URL:

2025-04-23 09:36:31.407 AEST [22678/T22709] DEBUG_API: Requested API URI: ::1 -> PUT /api/lists/https://v.firebog.net/hosts/Prigent-Malware.txt ? type=block (Content-Type application/json; charset=utf-8)
2025-04-23 09:36:31.407 AEST [22678/T22709] DEBUG_API: Received payload with size: 81
2025-04-23 09:36:31.408 AEST [22678/T22709] DEBUG_API: Read sid=[redacted] from cookie
2025-04-23 09:36:31.408 AEST [22678/T22709] DEBUG_API: Recognized known user: user_id 0, valid_until: 2025-04-23 10:06:31 AEST, remote_addr ::1 (::1 at login)
2025-04-23 09:36:31.408 AEST [22678/T22709] DEBUG_API: Processing PUT /api/lists/https://v.firebog.net/hosts/Prigent-Malware.txt in /api/lists
2025-04-23 09:36:31.428 AEST [22678/T22709] DEBUG_API: SQL: INSERT INTO adlist (address,enabled,comment,type) VALUES (:item,:enabled,:comment,:type) ON CONFLICT(address,type) DO UPDATE SET enabled = :enabled, comment = :comment, type = :type;
2025-04-23 09:36:31.428 AEST [22678/T22709] DEBUG_API:      :item = "https://v.firebog.net/hosts/Prigent-Malware.txt"
2025-04-23 09:36:31.428 AEST [22678/T22709] DEBUG_API:      :type = 0
2025-04-23 09:36:31.428 AEST [22678/T22709] DEBUG_API:      :comment = "Fabrice Prigent Malware"
2025-04-23 09:36:31.428 AEST [22678/T22709] DEBUG_API:      :enabled = 0
2025-04-23 09:36:31.429 AEST [22678/T22709] DEBUG_API: SQL: SELECT id FROM adlist WHERE address = :item AND type = :type
2025-04-23 09:36:31.429 AEST [22678/T22709] DEBUG_API:      :item = "https://v.firebog.net/hosts/Prigent-Malware.txt"
2025-04-23 09:36:31.429 AEST [22678/T22709] DEBUG_API:      :type = "0"
2025-04-23 09:36:31.440 AEST [22678/T22709] DEBUG_API: SQL: DELETE FROM adlist_by_group WHERE adlist_id = :id;
2025-04-23 09:36:31.440 AEST [22678/T22709] DEBUG_API:      :id = "21"
2025-04-23 09:36:31.441 AEST [22678/T22709] DEBUG_API: groupscount = 1
2025-04-23 09:36:31.455 AEST [22678/T22709] DEBUG_API: INSERT: 101 -> (21,0)
2025-04-23 09:36:31.455 AEST [22678/T22709] DEBUG_API: SQL: INSERT INTO adlist_by_group (adlist_id,group_id) VALUES (:id,:gid);
2025-04-23 09:36:31.455 AEST [22678/T22709] DEBUG_API:      :id = "21"
2025-04-23 09:36:31.456 AEST [22678/T22709] DEBUG_API:      :gid = "0"
2025-04-23 09:36:31.456 AEST [22678/T22709] DEBUG_API: SQL: SELECT id,type,address,enabled,date_added,date_modified,comment,(SELECT GROUP_CONCAT(group_id) FROM adlist_by_group g WHERE g.adlist_id = a.id) AS group_ids,date_updated,number,invalid_domains,status,abp_entries FROM adlist a WHERE type = 0 AND address = :item;
2025-04-23 09:36:31.457 AEST [22678/T22709] DEBUG_API:      :item = "https://v.firebog.net/hosts/Prigent-Malware.txt"
2025-04-23 09:36:31.457 AEST [22678/T22709] DEBUG_API:      :ids = "(null)"
2025-04-23 09:36:31.460 AEST [22678/T22709] DEBUG_API: Done

Sorry for all the bother, but glad to get it working.

system · May 13, 2025, 11:45pm

This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.