# Providers

### Amazon Web Services (AWS)

Amazon Web Services can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
  provider: aws
  # id is the name defined by user for filtering (optional)
  id: staging
  # aws_access_key is the access key for AWS account (optional - see "Keyless Authentication" below)
  aws_access_key: $AWS_ACCESS_KEY
  # aws_secret_key is the secret key for AWS account (optional - see "Keyless Authentication" below)
  aws_secret_key: $AWS_SECRET_KEY
  # aws_session_token session token for temporary security credentials retrieved via STS (optional)
  aws_session_token: $AWS_SESSION_TOKEN
  # assume_role_arn is the arn of the role to assume (optional)
  assume_role_arn: $AWS_ASSUME_ROLE_ARN
  # external_id is the external id for the role to assume (required if assume_role_arn is provided)
  external_id: $AWS_EXTERNAL_ID
  # assume_role_session_name is the name of the session for the role to assume (required if assume_role_arn is provided)
  assume_role_session_name: $AWS_ASSUME_ROLE_SESSION_NAME
 # assume_role_name is the name of the role to assume (optional)
 assume_role_name: $AWS_ASSUME_ROLE_NAME
 # account_ids is the aws account ids which has similar assumed role name (optional)
 account_ids:
   - $AWS_ACCOUNT_ID_1
   - $AWS_ACCOUNT_ID_2
```

`aws_access_key` and `aws_secret_key` can be generated in the IAM console. We recommend creating a new IAM user with `Read Only` permissions and providing the access token for the user.

#### Keyless Authentication (IRSA / Instance Profile / Environment)

`aws_access_key` and `aws_secret_key` are **optional**. When both are omitted from the config, the provider falls back to the AWS SDK default credential chain, so no secrets need to be injected into the config file. To use keyless mode, leave the keys out entirely (an `aws_access_key: $UNSET_ENV_VAR` whose environment variable is unset is treated as a literal value, not as keyless). This enables:

- **IRSA (IAM Roles for Service Accounts)** on EKS — credentials are picked up automatically from the `AWS_WEB_IDENTITY_TOKEN_FILE` and `AWS_ROLE_ARN` environment variables injected by the pod identity webhook.
- **EC2 / ECS instance profiles** — credentials are resolved from the instance/task metadata endpoint.
- **Environment variables** — `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`.
- **Shared config / profile** — `~/.aws/credentials` and `~/.aws/config`.

```yaml
# Zero-key config relying on IRSA / instance profile / environment
- provider: aws
  id: keyless-discovery
  # no aws_access_key / aws_secret_key — credentials resolved by the SDK default chain
```

> Note: if you provide one of `aws_access_key` / `aws_secret_key`, you must provide both. `assume_role_arn`, `assume_role_name` and `org_discovery_role_arn` all work on top of keyless base credentials too.

Scopes Required - 
1. EC2
2. Route53

References - 
1. https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_iam_read-only-console.html
2. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html
3. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html

- Aws Assume Role:

  - https://docs.aws.amazon.com/sdkref/latest/guide/feature-assume-role-credentials.html
  - https://docs.logrhythm.com/OCbeats/docs/aws-cross-account-access-using-sts-assume-role
   
### Google Cloud Platform (GCP)

Google Cloud Platform supports **two discovery approaches** and **two authentication modes**:

#### Authentication Modes

**A. Traditional Authentication (Static Credentials)**
- Service account keys (JSON files) - long-lived
- Application Default Credentials (ADC) - often long-lived

**B. Short-lived Credentials (Recommended for Enhanced Security)**
- Generate temporary access tokens (up to 1 hour)
- Eliminates reliance on static service account keys
- Uses Service Account Credentials API for token generation
- Supports service account impersonation

---

#### 1. Individual Service APIs (Project-Level Discovery)

**Option 1: Traditional Static Credentials**

```yaml
- # provider is the name of the provider
  provider: gcp
  # id is the name defined by user for filtering (optional)
  id: project-discovery
  # gcp_service_account_key is the key token of service account.
  gcp_service_account_key: '{
    "type": "service_account",
    "project_id": "your-project-id",
    "private_key_id": "...",
    "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
    "client_email": "cloudlist-sa@your-project-id.iam.gserviceaccount.com",
    "client_id": "...",
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/cloudlist-sa%40your-project-id.iam.gserviceaccount.com",
    "universe_domain": "googleapis.com"
  }'
```

**Option 2: Short-lived Credentials (Developer Workflow - Zero Keys)**

```yaml
- provider: gcp
  id: dev-discovery
  use_short_lived_credentials: true
  service_account_email: "cloudlist@project.iam.gserviceaccount.com"
  # Uses: gcloud auth login → ADC → short-lived token
  # No service account key file needed!
```

**Option 3: Short-lived Credentials (CI/CD with Minimal Permissions)**

```yaml
- provider: gcp
  id: ci-discovery
  use_short_lived_credentials: true
  service_account_email: "powerful-sa@project.iam.gserviceaccount.com"
  source_credentials: "minimal-ci-sa.json"  # Only has impersonation permission
  token_lifetime: "7200s"  # 2 hours
```

**Option 4: Short-lived Credentials (GKE/Compute Engine - Zero Secrets)**

```yaml
- provider: gcp
  id: workload-discovery
  use_short_lived_credentials: true
  service_account_email: "cloudlist@project.iam.gserviceaccount.com"
  # Uses workload identity automatically
```

**Option 5: Short-lived Credentials (Migration from Existing Keys)**

```yaml
- provider: gcp
  id: migrating-discovery
  use_short_lived_credentials: true
  service_account_email: "cloudlist@project.iam.gserviceaccount.com"
  gcp_service_account_key: "existing-key.json"  # Generates short-lived from static key
  token_lifetime: "3600s"  # 1 hour (default)
```

**Required Scopes (Traditional Authentication):**
1. `roles/compute.viewer` - Compute instances and forwarding rules
2. `roles/dns.reader` - DNS records
3. `roles/storage.objectViewer` - Storage buckets
4. `roles/run.viewer` - Cloud Run services
5. `roles/cloudfunctions.viewer` - Cloud Functions
6. `roles/container.viewer` - GKE clusters
7. `roles/tpu.viewer` - TPU nodes
8. `roles/file.viewer` - Filestore instances
9. `roles/resourcemanager.viewer` - List projects

**Additional Requirements for Short-lived Credentials:**
- **Source credentials** need: `roles/iam.serviceAccountTokenCreator` or `iam.serviceAccounts.generateAccessToken` permission on the target service account
- **Target service account** needs: Same viewer roles listed above
- **Service Account Credentials API** must be enabled in the project

**Configuration Parameters:**
- `use_short_lived_credentials` (bool): Enable short-lived token generation (default: false)
- `service_account_email` (string, required if short-lived): Target service account to impersonate
- `source_credentials` (string, optional): Path to source credentials file (uses ADC if not provided)
- `token_lifetime` (string, optional): Token lifetime in seconds (e.g., "3600s") or Go duration format (e.g., "1h"). Range: 1s to 3600s (1 hour). Default: "3600s"
- `project_ids` (list, optional): Comma-separated/list of project IDs to enumerate. When provided, Cloudlist skips discovery in every other accessible project, both for individual APIs and the organization-level Asset API.
- `exclude_project_ids` (list, optional): Comma-separated/list of project IDs to exclude from organization-wide discovery. Requires `organization_id`. Mutually exclusive with `project_ids`. When provided, Cloudlist discovers all projects in the organization and skips the excluded ones, using per-project Asset API calls for the remaining projects.

---

#### 2. Organization-Level Asset API (Organization-Wide Discovery)

**Traditional Authentication:**

```yaml
- # provider is the name of the provider
  provider: gcp
  # id is the name defined by user for filtering (optional)
  id: org-discovery
  # organization_id enables Asset API for organization-wide discovery
  organization_id: "123456789012"
  # gcp_service_account_key with organization-level permissions
  gcp_service_account_key: '{
    "type": "service_account",
    "project_id": "your-project-id",
    "private_key_id": "...",
    "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
    "client_email": "asset-viewer-sa@your-project-id.iam.gserviceaccount.com",
    "client_id": "...",
    "auth_uri": "https://accounts.google.com/o/oauth2/auth",
    "token_uri": "https://oauth2.googleapis.com/token",
    "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
    "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/asset-viewer-sa%40your-project-id.iam.gserviceaccount.com",
    "universe_domain": "googleapis.com"
  }'
```

**Short-lived Credentials (Organization-Level):**

```yaml
- provider: gcp
  id: org-discovery-secure
  organization_id: "123456789012"
  use_short_lived_credentials: true
  service_account_email: "asset-viewer-sa@project.iam.gserviceaccount.com"
  token_lifetime: "7200s"  # 2 hours
  project_ids:
    - security-core
    - shared-infra
```

Add `project_ids` to either configuration style to limit enumeration strictly to the listed projects (Cloud Asset API requests are filtered too), which is helpful for large organizations or delegated-access service accounts.

**Excluding Projects:**

```yaml
- provider: gcp
  organization_id: "123456789012"
  gcp_service_account_key: '$GCP_SA_KEY'
  exclude_project_ids:
    - sandbox-project
    - legacy-app
    - test-environment
```

Use `exclude_project_ids` to scan all projects in an organization except the listed ones. This is useful when only a few projects need to be excluded from a large organization.

**Required Organization-Level Roles:**
1. `roles/cloudasset.viewer` - Core Asset API access
2. `roles/resourcemanager.viewer` - List projects in organization
3. (For short-lived) `roles/iam.serviceAccountTokenCreator` - On source credentials

**Key Differences:**
- **Individual APIs**: Fast, project-specific, detailed results
- **Asset API**: Comprehensive, organization-wide, higher resource count
- **Short-lived Credentials**: Enhanced security, tokens auto-expire (up to 1 hour)

📚 **For detailed setup instructions, see: [docs/GCP_ASSET_API.md](docs/GCP_ASSET_API.md)**

References -
1. https://cloud.google.com/asset-inventory/docs/overview
2. https://cloud.google.com/iam/docs/creating-managing-service-accounts
3. https://cloud.google.com/iam/docs/understanding-roles
4. https://cloud.google.com/iam/docs/service-account-creds (Short-lived credentials)
5. https://cloud.google.com/docs/authentication/provide-credentials-adc (Application Default Credentials)


### Microsoft Azure

Microsoft Azure can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
 provider: azure
 # id is the name defined by user for filtering (optional)
 id: staging
 # client_id is the client ID of registered application of the azure account (not requuired if using cli auth)
 client_id: $AZURE_CLIENT_ID
 # client_secret is the secret ID of registered application of the zure account (not requuired if using cli auth)
 client_secret: $AZURE_CLIENT_SECRET
 # tenant_id is the tenant ID of registered application of the azure account (not requuired if using cli auth)
 tenant_id: $AZURE_TENANT_ID
 #subscription_id is the azure subscription id
 subscription_id: $AZURE_SUBSCRIPTION_ID
 #use_cli_auth if set to true cloudlist will use azure cli auth
 use_cli_auth: true
```

`tenant_id`, `client_id`, `client_secret` can be obtained/generated from   `All services` > `Azure Active Directory` > `App registrations`
`subscription_id` can be retrieved from  `All services` > `Subscriptions`

To use cli auth set `use_cli_auth` value to `true` and run `az login` in the terminal


References - 
1. https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli
2. https://docs.microsoft.com/en-us/cli/azure/ad/sp?view=azure-cli-latest#az_ad_sp_create_for_rbac
3. https://docs.microsoft.com/en-us/cli/azure/authenticate-azure-cli





   

### DigitalOcean (DO)

Digitalocean can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
  provider: do
  # digitalocean_token is the API key for digitalocean cloud platform
  digitalocean_token: $DIGITALOCEAN_TOKEN
```

`digitalocean_token` can be generated from the Digitalocean Control Panel. We recommend only giving Read Access to the token.

References - 
1. https://www.digitalocean.com/docs/apis-clis/api/create-personal-access-token/
   
### Scaleway (SCW)

Scaleway can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
  provider: scw
  # scaleway_access_key is the access key for scaleway API
  scaleway_access_key: $SCALEWAY_ACCESS_KEY
  # scaleway_access_token is the access token for scaleway API
  scaleway_access_token: $SCALEWAY_ACCESS_TOKEN
```

`scaleway_access_key` and `scaleway_access_token` can be generated from the Credentials Options in scaleway console.

References - 
1. https://www.scaleway.com/en/docs/generate-api-keys/
   
### ArvanCloud

ArvanCloud can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
  provider: arvancloud # or r1c
  # api_key is the api_key for arvancloud
  api_key: $R1C_API_KEY
```

The `api_key` can be generated from ArvanCloud Machine User manager.

References - 
1. https://www.arvancloud.ir/en/dev/api
2. https://docs.arvancloud.ir/en/developer-tools/api/api-usage
3. https://panel.arvancloud.ir/profile/machine-user

### Cloudflare 

Cloudflare can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
  provider: cloudflare
  # email is the email for cloudflare
  email: $CF_EMAIL
  # api_key is the api_key for cloudflare
  api_key: $CF_API_KEY
  # api_token is the scoped_api_token for cloudflare (optional)
  api_token: $CF_API_TOKEN
```

`api_key` can be generated from Cloudflare API Key manager. It needs to be Global API Key due to limitation of cloudflare new API tokens.

References - 
1. https://developers.cloudflare.com/api/keys
   
### Heroku

Heroku can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
 provider: heroku
 # id is the name defined by user for filtering (optional)
 id: staging
 # heroku_api_token is the authorization token for Heroku account
 heroku_api_token: $HEROKU_API_TOKEN
```

`heroku_api_token` can be generated from https://dashboard.heroku.com/account/applications/authorizations/new

It can also be created with the Heroku CLI by running:
```bash
$ heroku authorizations:create -d "brief description of token"
Creating OAuth Authorization... done
Client:      <none>
ID:          a6e98151-f242-4592-b107-25fbac5ab410
Description: brief description of token
Scope:       global
Token:       cf0e05d9-4eca-4948-a012-b9xxxxxxxxxx
Updated at:  Fri Jun 16 2021 13:26:56 GMT-0700 (PDT) (less than a minute ago)
```


References - 
1. https://devcenter.heroku.com/articles/platform-api-quickstart#authentication

### Fastly

Fastly can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
 provider: fastly
 # id is the name defined by user for filtering (optional)
 id: staging
 # fastly_api_key is the personal API token for fastly account
 fastly_api_key: $FASTLY_API_KEY
```

`fastly_api_key` can be generated from https://manage.fastly.com/account/personal/tokens

References - 
1. https://docs.fastly.com/en/guides/using-api-tokens#creating-api-tokens


### Linode

Linode can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
 provider: linode
 # id is the name defined by user for filtering (optional)
 id: staging
 # linode_personal_access_token is the personal access token for Linode account
 linode_personal_access_token: $LINODE_PERSONAL_ACCESS_TOKEN
```

`linode_personal_access_token` can be created from https://cloud.linode.com/id/tokens. Minimum scope needed is `Read Only` for `Linodes` resource.

References - 
1. https://www.linode.com/docs/guides/getting-started-with-the-linode-api/#get-an-access-token


### Namecheap

Namecheap can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
  provider: namecheap
  # id is the name defined by user for filtering (optional)
  id: staging
  # namecheap_api_key is the api key for namecheap account
  namecheap_api_key: $NAMECHEAP_API_KEY
  # namecheap_user_name is the username of the namecheap account
  namecheap_user_name: $NAMECHEAP_USER_NAME
```

Namecheap API Access can be enabled by visiting https://ap.www.namecheap.com/settings/tools/apiaccess/ and then:
- Toggle ON API Access switch
- Add your public IP to Whitelistted IPs 


References - 
- https://www.namecheap.com/support/api/intro/
    - Enabling API Access
    - Whitelisting IP


### Alibaba Cloud

Alibaba Cloud can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
 provider: alibaba
 # id is the name defined by user for filtering (optional)
 id: staging
 # alibaba_region_id is the region id of the resources
 alibaba_region_id: $ALIBABA_REGION_ID
 # alibaba_access_key is the access key ID for alibaba cloud account
 alibaba_access_key: $ALIBABA_ACCESS_KEY
 # alibaba_access_key_secret is the secret access key for alibaba cloud account
 alibaba_access_key_secret: $ALIBABA_ACCESS_KEY_SECRET
```

Alibaba Cloud Access Key ID and Secret can be created by visiting https://ram.console.aliyun.com/manage/ak


References - 
- https://www.alibabacloud.com/help/faq-detail/142101.htm
- https://www.alibabacloud.com/help/doc-detail/53045.htm 


### Terraform

Terraform can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
 provider: terraform
 # id is the name defined by user for filtering (optional)
 id: staging
 #tf_state_file is the location of terraform state file (terraform.tfsate) 
 tf_state_file: path/to/terraform.tfstate
```
### Hashicorp Consul

Hashicorp Consul can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
  provider: consul
  # consul_url is the url for consul server
  consul_url: http://localhost:8500/
  # id is the name defined by user for filtering (optional)
  id: staging
  # consul_ca_file is the path to consul CA file
  # consul_ca_file: <path-to-ca-file>.pem
  # consul_cert_file is the path to consul Certificate file
  # consul_cert_file: <path-to-cert-file>.pem
  # consul_key_file is the path to consul Certificate Key file
  # consul_key_file: <path-to-key-file>.pem
  # consul_http_token is the consul authentication token
  # consul_http_token: <consul-token>
  # consul_http_auth is the consul http auth value
  # consul_http_auth: <consul-http-auth-value>
```

Specifying https in the `consul_url` automatically turns SSL to on. All the fields are optional except the `consul_url`.

References - 
- https://www.consul.io/api-docs

### Hashicorp Nomad

Hashicorp Nomad can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
  provider: nomad
  # nomad_url is the url for nomad server
  nomad_url: http://127.0.0.1:4646/
  # id is the name defined by user for filtering (optional)
  id: staging
  # nomad_ca_file is the path to nomad CA file
  # nomad_ca_file: <path-to-ca-file>.pem
  # nomad_cert_file is the path to nomad Certificate file
  # nomad_cert_file: <path-to-cert-file>.pem
  # nomad_key_file is the path to nomad Certificate Key file
  # nomad_key_file: <path-to-key-file>.pem
  # nomad_token is the nomad authentication token
  # nomad_token: <nomad-token>
  # nomad_http_auth is the nomad http auth value
  # nomad_http_auth: <nomad-http-auth-value>

```

Specifying https in the `nomad_url` automatically turns SSL to on. All the fields are optional except the `nomad_url`.

References - 
- https://www.nomadproject.io/api-docs

### Hetzner Cloud

Hetzner Cloud can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
 provider: hetzner
 # id is the name defined by user for filtering (optional)
 id: staging
 # auth_token is the is the hetzner authentication token
 auth_token: $HETZNER_AUTH_TOKEN
```

References -
- https://docs.hetzner.cloud/#authentication

### Kubernetes

To integrate Kubernetes, utilize the configuration block outlined below. This block allows you to specify Kubernetes connection details either through a file path or directly by providing the encoded kubeconfig content. If both kubeconfig_file and kubeconfig_encoded are specified, the kubeconfig_encoded will take precedence.


```yaml
- # This identifies the provider in use
  provider: kubernetes
  # User-defined identifier for filtering (optional).
  id: staging
  # Path to the kubeconfig file.
  kubeconfig_file: path/to/kubeconfig
  # Base64 encoded kubeconfig, $ cat $KUBECONFIG | base64
  kubeconfig_encoded: <encoded-kubeconfig>
  # The context to use from the kubeconfig (optional). If omitted, the default is the current-context as defined in the kubeconfig.
  context: <context-name>  
```

References -
- https://www.redhat.com/sysadmin/kubeconfig
- https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html
- https://learn.microsoft.com/en-us/azure/aks/control-kubeconfig-access#get-and-verify-the-configuration-information
- https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl#store_info

### DNSSimple

DNSSimple can be integrated by using the following configuration block.

```yaml
- # provider is the name of the provider
  provider: dnssimple
  # id is the name defined by user for filtering (optional)
  id: main
  # dnssimple_api_token is the API token for DNSSimple
  dnssimple_api_token: $DNSSIMPLE_API_TOKEN
```

The `dnssimple_api_token` can be generated from the DNSSimple account settings under "API Access". You will need to create an API v2 token with appropriate permissions to access your domains and DNS records.

References - 
1. https://developer.dnsimple.com/v2/
2. https://support.dnsimple.com/articles/api-access-token/

### OVH

Cloudlist supports fetching DNS records from OVH.

- **Provider key**: `ovh`
- **Services**: `dns`
- **id**: An arbitrary label you choose to tag resources. It helps distinguish multiple OVH accounts/configs.
- **Required auth**:
  - `application_key`
  - `application_secret`
  - `consumer_key`
- **Endpoint**: OVH API endpoint. Defaults to `ovh-eu` if omitted. Common values: `ovh-eu`, `ovh-ca`, `ovh-us`.

Configuration example (from `ovh.yaml`):

```yaml
- provider: ovh
  id: ovh-prod
  endpoint: ovh-ca
  application_key: $OVH_APP_KEY
  application_secret: $OVH_APP_SECRET
  consumer_key: $OVH_CONSUMER_KEY
```

References - 
1. https://eu.api.ovh.com/console/?section=%2Fdomain&branch=v1
2. https://api.ovh.com/createToken/
3. https://help.ovhcloud.com/csm/en-gb-api-getting-started-ovhcloud-api
