PHP webscripts

PromoFaux · April 26, 2020, 5:34pm

I said don't quote me!

DanSchaper · April 26, 2020, 5:47pm

Whatever the outcome is, it needs to be sustainable. The code we have now is at a standstill because a single person knew it and worked on it. Bus Factor 1. We can't deploy something that depends on a single person. Development would grind to even slower than it is now. Support requests would only be answerable by the person that wrote it. That support was not going to be available for various reasons so things were shelved.

We would very very gladly accept any assistance you can provide, every single member of the development team came to the project via a single Pull Request. Every single member of the development team were users first. We are a community project, the code itself is really just software, Pi-hole IS the users.

ScrewGravity · May 27, 2020, 7:49am

Congratz all on the new pi-hole release! Very nice.

So behind the scenes I'm working hard on a POC with react that works with the actual pi-hole system. My focus is currently not on implementing all functionality in there with dummy data, but on getting a simple SPA with working log-in page on pi-hole.

I have some questions regarding the authentication mechanism. In PHP and "older" dashboards it's more common to work with sessions. In newer SPA it's more common to work with mechanisms like OAUTH2 and JWT tokens. Is there already something in place that can be used for this purpose? I noticed GitHub - civetweb/civetweb: Embedded C/C++ web server doesnt have a lot of support for proper authentication systems so seems like a lot of custom logic is required for the REST endpoints and Authentication. I can also dive in the server-side things but I don't know my way around in your code and structure of authentication.

Idealy I'm used to a JWT Authentication system with 2 endpoints like:
/api/login
/api/register

Where the /api/login endpoint can be used to verify the JWT token. Next all REST calls to Pi-Hole will have a token in the header so the server can validate if the user is authenticated with the claimset. https://jwt.io/

Coro · May 27, 2020, 9:43am

I see that the API proposed here is still being worked on (it was updated 11 hours ago):

github.com/pi-hole/FTL

Add HTTP server and JSON API

pi-hole:development ← pi-hole:new/http

opened 12:13PM - 21 Nov 19 UTC

DL6ER

+122183 -10306

**By submitting this pull request, I confirm the following (please check boxes, …eg [X]) _Failure to fill the template will close your PR_:** ***Please submit all pull requests against the `development` branch. Failure to do so will delay or deny your request*** - [X] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md). - [X] I have checked that [another pull request](https://github.com/pi-hole/FTL/pulls) for this purpose does not exist. - [X] I have considered, and confirmed that this submission will be valuable to others. - [X] I accept that this submission may not be used, and the pull request closed at the will of the maintainer. - [X] I give this submission freely, and claim no ownership to its content. **How familiar are you with the codebase?:** ## 10 --- This PR adds a fully-fledged HTTP server and JSON API to FTL. It will replace `lighttpd` and serve both the API and the [NG web interface](https://github.com/pi-hole/web). More detail is to be added here once this feature advances further.

Maybe @ScrewGravity and @DL6ER can coordinate? I was looking at the code of this new FTL API some days ago and I think it great what they are working on. Especially that they are doing server-side pagination. It is worth to check out the individual commits, @DL6ER is always describing what he adds and you can learn a lot from him. Everything is separated into small pieces with clear titles so there are no huge +/- 1000 lines as you can find in many other open source projects where everything is squashed into big commits, defeating the idea of commits IMHO somewhat.
Take, for instance f241287 Add /admin/api/stats/database/overTime/history - this allows users to select a different time interval on the dash board than only "last 24 hours".

When you can work on this together, I think this can be the way to go. Fitting new things to the old (and conceptually at least questionable in many aspects!) API seems wrong when there is already something a new RESTy in the works.

What do you think @ScrewGravity?

ScrewGravity · May 27, 2020, 10:48am

Hi Coro,

I agree the work from the pi-hole devs is amazing, can't thank them enough right. Server-side pagination also sounds good, love it!

However.. when moving away from PHP to a SPA we have to tackle the authentication topic. I'm missing the planning and alignment on this one so therefore I'm asking. I would like to help out and implement JWT in the back-end if we all agree on using that. We need something there to get the React SPA going right? It's just that I would rather focus fully on the front-end than also diving in the back-end if possible. The embeded web framework also doesnt offer a lot of functionality so seems like lot of work by hand there. I'm also a full-stack java developer but not so used to C, C#, bash etc.

Coro · May 27, 2020, 11:20am

I do think they have authentication already done?

https://github.com/pi-hole/FTL/blob/new/http/src/api/auth.c

The endpoint seems to be /api/auth

github.com

pi-hole/FTL/blob/8afd0631c1ffacca8f0041721e981f52b607f92f/src/api/routes.c#L148-L155


      
          	else if(startsWith("/api/auth", request->local_uri))
          	{
          		ret = api_auth(conn);
          	}
          	else if(startsWith("/api/auth/salt", request->local_uri))
          	{
          		ret = api_auth_salt(conn);
          	}

I tried the web interface myself some time ago (maybe January) and I recall having used a password to log in.

I don't think use registration should be added. This will soon raise the demand for multiple access levels (viewer, list maintainer, admin, etc.) and this will surely be too much to be done at the same time. This could still come later.

ScrewGravity · May 27, 2020, 12:17pm

Ok thanks, I saw there is also one based on the RUST programming language? The authentication mechanism looks very flawed but ok will try that one, thanks for helping me out!

PromoFaux · May 27, 2020, 12:20pm

The Rust one is archived. It was an avenue we were exploring but it has been shelved.

The new/http branch of FTL that @Coro just linked to above is the current focus for the future of the API.

github.com/pi-hole/FTL

Add HTTP server and JSON API

pi-hole:development ← pi-hole:new/http

opened 12:13PM - 21 Nov 19 UTC

DL6ER

+88672 -8704

**By submitting this pull request, I confirm the following (please check boxes, …eg [X]) _Failure to fill the template will close your PR_:** ***Please submit all pull requests against the `development` branch. Failure to do so will delay or deny your request*** - [X] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md). - [X] I have checked that [another pull request](https://github.com/pi-hole/FTL/pulls) for this purpose does not exist. - [X] I have considered, and confirmed that this submission will be valuable to others. - [X] I accept that this submission may not be used, and the pull request closed at the will of the maintainer. - [X] I give this submission freely, and claim no ownership to its content. **How familiar are you with the codebase?:** ## 10 --- This PR adds a fully-fledged HTTP server and JSON API to FTL. It will replace `lighttpd` and serve both the API and the next-gen web interface. More detail is to be added here once this feature advances further.

That was DL pulling in the latest changes from dev to keep it from going stale. Current thinking on this one is that it will be for 6.0... we still have 5.1+ to go before then

There is some limited functionality in the PHP API, some of which was (accidentally) removed in 5.0, but looking to restore it in 5.1:

github.com/pi-hole/web

(Re-)Add API endpoints for list manipulations

pi-hole:devel ← pi-hole:new/api_add_sub_get_lists

opened 04:39PM - 26 May 20 UTC

DL6ER

+137 -140

**By submitting this pull request, I confirm the following:** - [X] I have r…ead and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md), as well as this entire template. - [X] I have made only one major change in my proposed changes. - [X] I have commented my proposed changes within the code. - [X] I have tested my proposed changes, and have included unit tests where possible. - [X] I am willing to help maintain this change if there are issues with it later. - [X] I give this submission freely and claim no ownership. - [X] It is compatible with the [EUPL 1.2 license](https://opensource.org/licenses/EUPL-1.1) - [X] I have squashed any insignificant commits. ([`git rebase`](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html)) --- **What does this PR aim to accomplish?:** - Fix #1297 - Finally get rid of the remaining `add.php` and use the `groups.php` routines for everything (eases maintenance). **How does this PR accomplish the above?:** Add API endpoints `api.php?list=black&add=domain.com` like commands where `add=domain` and `sub=domain` can be used to manage the lists. A simple `api.php?list=black` will simply list all blacklisted domains. `list` can be `black`, `white`, `regex_white` and `regex_black`. **What documentation changes (if any) are needed to support this PR?:** None

DanSchaper · May 27, 2020, 1:26pm

Feel free to borrow or improve anything you find in GitHub - pi-hole/web: Pi-hole Dashboard for stats and more

DL6ER · May 27, 2020, 4:23pm

"Flawed"?

DL6ER · May 28, 2020, 10:38pm

Just a heads up: You may have seen my most recent comment on this branch. As it does not feasible to pursue a React-based Pi-hole v6.0 dashboard, I added PHP support into FTL's embedded web server.
You can already host the existing web interface as simple as

pihole checkout ftl new/http

and accessing the web interface at http://pi.hole:8080/admin. You may need to set the php-cgi path, however, the standard path should be fine for most setups. Note that, due to substantial changes to the API, many parts do not work at this time, however, there will be work going into this piece by piece.

This means that the branch is now officially revived and will lead the path into Pi-hole v6.0 (this is still a longer-term goal, ETA currently not before end of the year and maybe even afterwards).

ScrewGravity · May 28, 2020, 11:18pm

Ok!

ScrewGravity · May 29, 2020, 7:14am

@DL6ER what is the vision for the v6.0 without react? I noticed you saying that PHP is enabled again and I noticed you saying the you would like to ditch api.php. Can I safely assume that in 6.0 PHP will still be there with adminLTE? So in that case maybe I should work on getting the PHP codebase better for v6.0 and focus on that part completely?

DL6ER · May 29, 2020, 6:15pm

Pi-hole v6.0 will somewhere far in the future, think maybe on the order of a year. Nothing is decided, either. There is a lot we can still do with v5.x

The issue with waiting on React is that it may never come. I don't know how to code React and I think nobody from the "core" team knows it well enough to bring it to feature completion. Hence, my idea to bring the current web interface into this branch.

Removing api.php only makes sense as we'd otherwise have a man-in-the-middle. Being a PHP man, this will be slow. Instead, we modify our Javascripts to talk to the REST API endpoints provided by FTL itself, directly. I'm just finishing the porting of the dashboard index page (with all the graphs) to the new API endpoints so you can see what is involved here.