Allow to configure mirror public access

.env

@@ -50,3 +50,6 @@ APP_COMPOSER_HOME="%kernel.project_dir%/var/.composer"
 ###> nelmio/cors-bundle ###
 CORS_ALLOW_ORIGIN='^https?://(localhost|127\.0\.0\.1)(:[0-9]+)?$'
 ###< nelmio/cors-bundle ###
+
+TRUSTED_HOSTS=
+TRUSTED_PROXIES=172.16.0.0/12

README.md

@@ -19,9 +19,9 @@ Features
 --------
 
 - Compatible with Composer API v2, bases on Symfony 5.4.
-- Support update webhook for GitHub, Bitbucket and GitLab or custom format.
+- Support update webhook for GitHub, Gitea, Bitbucket and GitLab or custom format.
 - Customers user and ACL groups and limit access by vendor and versions.
-- Composer Proxies. [docs](docs/usage/mirroring.md)
+- Composer Proxies and Mirroring.
 - Generic Packeton [webhooks](docs/webhook.md)
 - Allow to freeze updates for the new releases after expire a customers license.
 - Mirroring for packages zip files and downloads it's from your host.
@@ -54,6 +54,7 @@ Table of content
     - [Bitbucket](#bitbucket-webhooks)
     - [Manual hook](#manual-hook-setup)
     - [Custom webhook format](#custom-webhook-format-transformer)
+- [Mirroring Composer repos](docs/usage/mirroring.md)
 - [Usage](#usage-and-authentication)
     - [Create admin user](#create-admin-user)
 
@@ -104,6 +105,7 @@ docker-compose up -f docker-compose-prod.yml -d # Or split
 - `REDIS_URL` - Redis DB, default redis://localhost
 - `PACKAGIST_DIST_HOST` - Hostname, (auto) default use the current host header in the request.
 - `TRUSTED_PROXIES` - Ips for Reverse Proxy. See [Symfony docs](https://symfony.com/doc/current/deployment/proxies.html)
+- `TRUSTED_HOSTS` - Trusted host, set if you've enabled public access and your nginx configuration uses without `server_name`. Otherwise, possible the DDoS attack with generated a big cache size for each host.
 - `PUBLIC_ACCESS` - Allow anonymous users access to read packages metadata, default: `false`
 - `MAILER_DSN` - Mailter for reset password, default disabled
 - `MAILER_FROM` - Mailter from
@@ -246,8 +248,16 @@ it would with any other git repository. You can enable it again with env option
 
 Update Webhooks
 ---------------
-You can use GitLab, GitHub, and Bitbucket project post-receive hook to keep your packages up to date 
-every time you push code.
+You can use GitLab, Gitea, GitHub, and Bitbucket project post-receive hook to keep your packages up to date 
+every time you push code. More simple way use group webhooks, to prevent from being added it per each repository manually.
+
+| Provider  | Group webhook support | Target Path                                               |
+|-----------|-----------------------|-----------------------------------------------------------|
+| GitHub    | Yes                   | `https://example.org/api/github?token=`                   |
+| GitLab    | Only paid plan        | `https://example.org/api/update-package?token=`           |
+| Gitea     | Yes                   | `https://example.org/api/update-package?token=`           |
+| Bitbucket | Yes                   | `https://example.org/api/bitbucket?token=`                |
+| Custom    | -                     | `https://example.org/api/update-package/{packnam}?token=` |
 
 #### Bitbucket Webhooks
 To enable the Bitbucket web hook, go to your BitBucket repository, 
@@ -273,11 +283,11 @@ Enter `https://<app>/api/update-package?token=user:token` as URL.
 To enable the GitHub webhook go to your GitHub repository. Click the "Settings" button, click "Webhooks". 
 Add a new hook. Enter `https://<app>/api/github?token=user:token` as URL.
 
-#### Manual hook setup
+#### Manual or other hook setup
 
 If you do not use Bitbucket or GitHub there is a generic endpoint you can call manually 
 from a git post-receive hook or similar. You have to do a POST request to 
-`https://pkg.okvpn.org/api/update-package?token=user:api_token` with a request body looking like this:
+`https://example.org/api/update-package?token=user:api_token` with a request body looking like this:
 
 ```
 {
@@ -287,24 +297,18 @@ from a git post-receive hook or similar. You have to do a POST request to
 }
 ```
 
-Also, you can overwrite regex that was used to parse the repository url, 
-see [ApiController](src/Controller/ApiController.php#L348)
+It will be works with Gitea by default.
+
+Also, you can use package name in path parameter, see [ApiController](src/Controller/ApiController.php#L78)
 
 ```
-{
-  "repository": {
-    "url": "PACKAGIST_PACKAGE_URL"
-  },
-  "packeton": {
-    "regex": "{^(?:ssh://git@|https?://|git://|git@)?(?P<host>[a-z0-9.-]+)(?::[0-9]+/|[:/])(scm/)?(?P<path>[\\w.-]+(?:/[\\w.-]+?)+)(?:\\.git|/)?$}i"
-  }
-}
+https://example.org/api/update-package/acme/packet1?token=<token>
 ```
 
 You can do this using curl for example:
 
 ```
-curl -XPOST -H 'content-type:application/json' 'https://pkg.okvpn.org/api/update-package?token=user:api_token' -d' {"repository":{"url":"PACKAGIST_PACKAGE_URL"}}'
+curl -XPOST -H 'content-type:application/json' 'https://example.org/api/update-package?token=user:api_token' -d' {"repository":{"url":"PACKAGIST_PACKAGE_URL"}}'
 ```
 
 Instead of using repo url you can use directly composer package name. 
@@ -329,7 +333,7 @@ You have to do a POST request with a request body.
 #### Custom webhook format transformer
 
 You can create a proxy middleware to transform JSON payload to the applicable inner format.
-In first you need create a new Rest Endpoint to accept external request.
+In the first you need create a new Rest Endpoint to accept external request.
 
 Go to `Settings > Webhooks` and click `Add webhook`. Fill the form:
  - url - `https://<app>/api/update-package?token=user:token`
@@ -386,7 +390,7 @@ The customer users can only see related packages and own profile with instructio
 To authenticate composer access to repository needs add credentials globally into auth.json, for example:
 
 ```
-composer config --global --auth http-basic.pkg.okvpn.org <user> <token>
+composer config --global --auth http-basic.example.org <user> <token>
 ```
 
 API Token you can found in your Profile.

config/packages/framework.yaml

@@ -5,6 +5,7 @@ framework:
     http_method_override: false
     trusted_proxies: '%env(TRUSTED_PROXIES)%'
     trusted_headers: ['x-forwarded-for', 'x-forwarded-host', 'x-forwarded-proto', 'x-forwarded-port', 'x-forwarded-prefix']
+    trusted_hosts: '%env(TRUSTED_HOSTS)%'
 
     cache:
         pools:

config/packages/nelmio_security.yaml

@@ -24,7 +24,8 @@ nelmio_security:
                 - 'self'
             script-src:
                 - 'self'
-                - 'https://cdn.jsdelivr.net'
+                - 'https://cdn.jsdelivr.net/npm/[email protected]/'
+                - 'https://cdn.jsdelivr.net/npm/[email protected]/'
             connect-src:
                 - 'self'
             img-src:

config/packages/security.yaml

@@ -60,7 +60,8 @@ security:
         # Packagist
         - { path: (^(/change-password|/profile|/logout))+, roles: ROLE_USER }
         - { path: (^(/search|/packages/|/versions/))+, roles: ROLE_USER, allow_if: "is_granted('PACKETON_PUBLIC')" }
-        - { path: (^(/packages.json$|/p/|/p2/|/mirror/|/downloads/))+, roles: ROLE_USER, allow_if: "is_granted('PACKETON_PUBLIC')" }
+        - { path: ^/mirror/, roles: ROLE_USER, allow_if: "is_granted('PACKETON_MIRROR_PUBLIC')" }
+        - { path: (^(/packages.json$|/p/|/p2/|/downloads/))+, roles: ROLE_USER, allow_if: "is_granted('PACKETON_PUBLIC')" }
         - { path: (^(/zipball/))+, roles: ROLE_USER, allow_if: "is_granted('PACKETON_ARCHIVE_PUBLIC')" }
         - { path: (^(/api/webhook-invoke/))+, roles: ROLE_USER }
         - { path: (^(/api/(create-package|update-package|github|bitbucket)|/apidoc|/about))$, roles: ROLE_MAINTAINER }

config/services.yaml

@@ -98,6 +98,7 @@ services:
         arguments:
             $isAnonymousAccess: '%anonymous_access%'
             $isAnonymousArchiveAccess: '%anonymous_archive_access%'
+            $isAnonymousMirror: '%anonymous_mirror_access%'
 
     Packeton\Service\DistConfig:
         arguments:

docker-compose.yml

@@ -12,6 +12,8 @@ services:
             ADMIN_PASSWORD: 123456
             ADMIN_EMAIL: [email protected]
             TRUSTED_PROXIES: 172.16.0.0/12
+            # Default SQLite
+            # DATABASE_URL: "mysql://app:[email protected]:3306/app?serverVersion=8&charset=utf8mb4"
         ports:
             - '127.0.0.1:8088:80'
         volumes:

docs/usage/mirroring.md

@@ -1,18 +1,19 @@
 # Mirroring and Composer proxies
 
-Packeton may work as a proxy for the composer's repository, including requiring an authorization.
-This can be used to give all developers and clients access to private repositories like Magento.
-Also, possible to create zip archives from git repositories of mirroring packages, if http dist is not available.
+Packeton can function as a proxy for the Composer repository, including requiring an authorization.
+This feature can be used to grant all developers and clients access to private repositories such as Magento.
+Additionally, it is possible to create ZIP archives from mirrored Git repositories of packages, in cases where HTTP dist
+is unavailable.
 
 Main Features
 -------------
 
-- Support full and lazy sync for all - smail and big composer repositories.
-- Support Packagist fast `metadata-changes-url` API.
-- Strict mode and Dependencies approvement.
-- Dist mirroring
+- Supports full and lazy synchronization for small and large Composer repositories.
+- Supports the Packagist fast `metadata-changes-url` API.
+- Includes Strict Mode and Dependencies Approval functionality.
+- Supports Dist/SSH mirroring of source code.
 
-Example metadata with Strict mode and manual dependencies approvement.
+Example metadata with Strict mode and manual dependencies' approval.
 
 ```json
 {
@@ -61,15 +62,16 @@ Original metadata is:
     + 57 packages
 ```
 
-For performance if composer user-agent is not 1 we remove `includes` and use `providers-lazy-url`
+For performance if composer user-agent == 1 then `includes` replaced with `providers-lazy-url`
 
 
 [![logo](../img/packeton_proxies.png)](../img/packeton_proxies.png)
 
 ## Configuration
 
 Example how to enable proxies in your local configuration.
-Create a file `config/packages/any-name.yaml` with config.
+To enable proxies in your local configuration, create a file with any name
+like `config/packages/any-name.yaml` and add the following configuration:
 
 ```yaml
 packeton:
@@ -87,20 +89,33 @@ packeton:
             http_basic:
                 username: 123
                 password: 123
+            public_access: true # Allow public access, default false
             sync_lazy: true # default false 
             enable_dist_mirror: false # default true
             available_package_patterns: # Additional restriction, but you can restrict it in UI
                 - 'vend1/*' 
             available_packages:
                 - 'pack1/name1' # but you can restrict it in UI
-            composer_auth: '{"..."}' # JSON. auth.json to pass composer opts.
+            composer_auth: '{"auth.json..."}' # JSON. auth.json to pass composer opts.
             sync_interval: 3600 # default auto.
             info_cmd_message: "\n\u001b[37;44m#Слава\u001b[30;43mУкраїні!\u001b[0m\n\u001b[40;31m#Смерть\u001b[30;41mворогам\u001b[0m" # Info message
 ```
 
-## Metadata proxy specification.
+The configuration allows you to use multiple SSH key settings for different GitHub accounts.
 
-It depends on the type of repository and sync strategy.
+```
+...
+git_ssh_keys:
+    [email protected]:oroinc: '/var/www/.ssh/private_key1'
+    [email protected]:org2: '/var/www/.ssh/private_key2'
+
+# Or one key
+git_ssh_keys: '/var/www/.ssh/private_key1'
+```
+
+## Metadata Proxy Specification.
+
+The specification for the metadata proxy depends on the type of repository and the synchronization strategy being used.
 
 | API | Full sync                                      | Lazy sync          | Mirroring (strict)           |
 |-----|------------------------------------------------|--------------------|------------------------------|
@@ -136,17 +151,31 @@ Options:
   -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
 ```
 
-## Manual approve dependencies
+## Manual Approval of Dependencies
 
-By default, all new packages are automatically enabled and added to your repository then you run composer update. But you can 
-enable strict mode to use only approved packages and avoid including into metadata untrusted packages.
-This can be useful to prevent dependency confusion attacks, for example if you use 3-d party composer repo like this `https://satis.oroinc.com/`
-See about [dependency confusion](https://blog.packagist.com/preventing-dependency-hijacking) 
+By default, all new packages are automatically enabled and added to your repository when you run composer update. 
+However, you can enable strict mode to use only approved packages and avoid including untrusted packages in your metadata. 
+This can be useful in preventing dependency confusion attacks, especially if you use a 3rd-party Composer repository 
+like `https://satis.oroinc.com/`. For more information on preventing dependency hacking, please see [dependency confusion](https://blog.packagist.com/preventing-dependency-hijacking) 
 
-To enable strict mode go to proxy settings page Composer proxies -> Packagist (or any name) -> Settings
+To enable strict mode, go to the Proxy Settings page and select Composer Proxies -> Packagist (or any other name) -> Settings.
 
 [![strict](../img/mirr1.png)](../img/mirr1.png)
 
-Next go to view proxy page and click "Mass mirror packages" button
+Next, go to the View Proxy page and click the "Mass Mirror Packages" button.
 
 [![strict](../img/mirr2.png)](../img/mirr2.png)
+
+## Mirror Public Access
+
+Use the following configuration:
+
+```yaml
+packeton:
+    mirrors:
+        youname:
+            url: https://repo.example.org
+            public_access: true
+```
+
+[![strict](../img/mirr2.png)](../img/mirr3.png)

src/Composer/Cache/MetadataCache.php

@@ -0,0 +1,48 @@
+<?php
+
+declare(strict_types=1);
+
+namespace Packeton\Composer\Cache;
+
+use Symfony\Component\HttpFoundation\RequestStack;
+use Symfony\Contracts\Cache\CacheInterface;
+
+class MetadataCache
+{
+    public function __construct(
+        private readonly RequestStack $requestStack,
+        private readonly CacheInterface $packagesCachePool,
+        private readonly int $maxTtl = 1800 // TTL default / 2
+    ) {
+    }
+
+    public function get(string $key, callable $callback, int $lastModify = null)
+    {
+        // Use host key to prevent Cache Poisoning attack, if dist URL generated dynamic.
+        // But for will protection must be used trusted_hosts
+        $httpKey = $this->requestStack->getMainRequest()?->getSchemeAndHttpHost();
+
+        $cacheKey = sha1($key . $httpKey);
+        $item = $this->packagesCachePool->getItem($cacheKey);
+        @[$ctime, $data] = $item->get();
+
+        $needRefresh = false;
+        if ($lastModify !== null) {
+            $needRefresh = $ctime < $lastModify || $ctime + $this->maxTtl < time();
+        }
+
+        if (!$item->isHit() || $needRefresh || empty($data)) {
+            $data = $callback($item);
+
+            $item->set([time(), $data]);
+            $this->packagesCachePool->save($item);
+        }
+
+        return $data;
+    }
+
+    public function delete(string $key): bool
+    {
+        return $this->packagesCachePool->delete($key);
+    }
+}

src/Controller/MirrorController.php

@@ -34,14 +34,14 @@ public function index(string $alias): Response
     {
         try {
             $this->checkAccess($alias);
-            $this->proxyRegistry->createRepository($alias);
+            $config = $this->proxyRegistry->getProxyConfig($alias);
         } catch (MetadataNotFoundException $e) {
             throw $this->createNotFoundException($e->getMessage(), $e);
         }
 
         $repo = $this->generateUrl('mirror_index', ['alias' => $alias], UrlGeneratorInterface::ABSOLUTE_URL);
 
-        return $this->render('proxies/mirror.html.twig', ['alias' => $alias, 'repoUrl' => $repo]);
+        return $this->render('proxies/mirror.html.twig', ['alias' => $alias, 'repoUrl' => $repo, 'repo' => $config]);
     }
 
     #[Route('/{alias}/packages.json', name: 'mirror_root', methods: ['GET'])]
@@ -136,7 +136,6 @@ protected function wrap404Error(string $alias, callable $callback): JsonMetadata
         try {
             $this->checkAccess($alias);
             $repo = $this->proxyRegistry->createACLAwareRepository($alias);
-
             return $callback($repo);
         } catch (MetadataNotFoundException $e) {
             throw $this->createNotFoundException($e->getMessage(), $e);
@@ -145,9 +144,20 @@ protected function wrap404Error(string $alias, callable $callback): JsonMetadata
 
     protected function checkAccess(string $alias)
     {
-        // ROLE_ADMIN have access to all proxies views
-        if (!$this->isGranted('ROLE_ADMIN') && !$this->isGranted('VIEW', new ObjectIdentity($alias, PRI::class))) {
-            throw $this->createAccessDeniedException();
+        try {
+            $config = $this->proxyRegistry->getProxyConfig($alias);
+        } catch (MetadataNotFoundException) {
+            throw $this->createNotFoundException();
+        }
+
+        if (false === $config->isPublicAccess()) {
+            if (null !== $this->getUser()) {
+                if (!$this->isGranted('ROLE_MAINTAINER') && !$this->isGranted('VIEW', new ObjectIdentity($alias, PRI::class))) {
+                    throw $this->createAccessDeniedException();
+                }
+            } else {
+                throw $this->createNotFoundException();
+            }
         }
     }
 }

src/DependencyInjection/Configuration.php

@@ -109,8 +109,10 @@ private function addMirrorsRepositoriesConfiguration(ArrayNodeDefinition|NodeDef
                 ->booleanNode('enable_dist_mirror')->defaultTrue()->end()
                 ->booleanNode('parent_notify')->end()
                 ->booleanNode('disable_v1')->end()
+                ->booleanNode('public_access')->end()
                 ->variableNode('git_ssh_keys')->end()
                 ->scalarNode('info_cmd_message')->end()
+                ->scalarNode('without_path_prefix')->end()
                 ->scalarNode('logo')->end()
                 ->integerNode('available_packages_count_limit')->end()
                 ->arrayNode('available_package_patterns')