The Weka cluster on Azure provides a fast and scalable platform to run, for example, performance-intensive applications and hybrid cloud workflows. It can also be used for object stores, tiering, and snapshots using the Azure Blob service. The provided Azure-Weka Terraform package contains modules and examples you can customize according to your deployment needs. The installation is based on applying the customized Terraform variables file to a predefined Azure subscription. Applying the Terraform variables file performs the following:
- Creates resources in a predefined resource group, such as virtual machines, network interfaces, function app, load balancer, and more.
- Deploys Azure virtual machines.
- Installs the Weka software.
- Configures the Weka cluster.
You can find here our essential deployment which creates only vms and placement group.
- resource group for deployment
- vnet
- subnet
- 2 subnets delegations - one for our function app and one for our logic app
- security group (needs to allow network inside the vnet)
- dns zone
We have 3 variables that define resource group:
- rg_name
- vnet_rg_name
- private_dns_rg_name
The resource group were weka cluster and all necessary resources will be deployed.
The resource group of the vnet and subnet.
The private DNS zone resource group name.
If vnet_rg_name
isn't set by the user, we assume that the
vnet and subnet resource group is the as the weka deployment resource group.
i.e we assume vnet_rg_name = rg_name
Same goes for private_dns_rg_name
.
If private_dns_rg_name
isn't set by the user, we assume that the
private dns resource group name is the same as the weka deployment resource group.
i.e we assume private_dns_rg_name = rg_name
This weka deployment can use existing network, or create network resources (vmet, subnet, security group etc.) automatically.
Check our examples.
In case you want to use an existing vnet and subnet, you must provide them.
Example:
vnet_name = "my-vnet"
subnet_name = "my-subnet"
In case you want to use an existing subnet delegations, you must provide them.
Example:
function_app_subnet_delegation_id = "subnet-delegation-id1"
logic_app_subnet_delegation_id = "subnet-delegation-id2"
In case you want to use an existing security group, you must provide it.
Example:
sg_id = "sg-id"
In case you want to use a dns zone, you must provide it.
Example:
private_dns_zone_name = "myDns.private.net"
private_dns_rg_name = "myResourceGroup"
If you don't pass these params, we will automatically create the network resources for you.
- Logic app storage account - Stores the logic app configuration. Created by our module.
- Deployment storage account - Stores the deployment states (cluster and also NFS if configured). Created by our module if not provided.
- Weka OBS storage account - Created by our function app if OBS is configured and OBS storage account is not provided.
variable "storage_account_public_network_access" {
type = string
description = "Public network access to the storage accounts."
default = "Enabled"
validation {
condition = contains(["Enabled", "Disabled", "EnabledForVnet"], var.storage_account_public_network_access)
error_message = "Allowed values: [\"Enabled\", \"Disabled\", \"EnabledForVnet\"]."
}
}
Enabled
: By default, the storage account is created with public network access enabled.EnabledForVnet
: The storage account is created with public network access enabled, but only for the specified virtual network.- Access should be enabled for the vnet, function app subnet delegation.
- File share is required (can provide existing via
deployment_file_share_name
or it will be auto-created in case ifstorage_account_allowed_ips
are provided). storage_account_allowed_ips
: required to allow creating the logic app storage account with the required config and function app file share.- if
storage_account_allowed_ips
if not provided, scale down and autoscaling will not be supported and the file share needs to be created by the user. - OBS storage account: if created by our module only the regular OBS config is required. If provided by the user needs to have the Vnet enabled.
Disabled
: The storage account is created with public network access disabled.- Scale down and autoscaling is not supported.
- Pre created deployment storage account is required.
- File share is required (
deployment_file_share_name
). - Blob and file endpoints and private links are required. It can be created by our module if
create_storage_account_private_links
is provided or by the user. In case if there are existing private endpoints andcreate_storage_account_private_links
is not set,storage_blob_private_dns_zone_name
can be also set to specify private DNS zone for blob resource (uses Azure-recommended name as default value). - OBS storage account: if created by our module only the regular OBS config is required. If provided by the user, blob and file endpoints and private links are required.
They can be created by our module if
create_storage_account_private_links
is provided.
provider "azurerm" {
subscription_id = "mySubscriptionId"
partner_id = "f13589d1-f10d-4c3b-ae42-3b1a8337eaf1"
features {
}
}
terraform {
backend "azurerm" {
resource_group_name = "myStateResourceGroup"
storage_account_name = "myStateStorageAccount"
container_name = "myStateContainer"
key = "state.terraform.tfstate"
}
}
module "deploy_weka" {
source = "weka/weka/azure"
version = "3.0.5"
prefix = "weka"
rg_name = "myResourceGroup"
vnet_name = "weka-vpc-0"
vnet_rg_name = "myVnetResourceGroup"
subnet_name = "weka-subnet-0"
sg_id = "security-group-id"
get_weka_io_token = "get_weka_io_token"
cluster_name = "myCluster"
function_app_subnet_delegation_id = "subnet-delegation-id1"
logic_app_subnet_delegation_id = "subnet-delegation-id2"
set_obs_integration = true
instance_type = "Standard_L8s_v3"
cluster_size = 6
assign_public_ip = false
subscription_id = "mySubscriptionId"
private_dns_zone_name = "myDns.private.net"
private_dns_rg_name = "myResourceGroup"
}
output "deploy_weka_output" {
value = module.deploy_weka
}
assign_public_ip = true
In case your vms don't have internet access, you should supply weka tar file url, apt repo url and service endpoints:
apt_repo_url = "..."
install_weka_url = "..."
The deployment and delegation subnets must include the following service endpoints:
- "Microsoft.Storage"
- "Microsoft.KeyVault"
- "Microsoft.Web"
The delegation subnets must include the following action action:
service_delegation {
name = "Microsoft.Web/serverFarms"
actions = ["Microsoft.Network/virtualNetworks/subnets/action"]
}
As you can see via source_image_id
variable, we use our own custom image.
This is a community image that we created and uploaded to azure.
In case you would like to view how we created the image you can find it here.
You can as well create it on your own subscription and use it.
The username for ssh into vms is weka
.
We allow passing an existing public key:
ssh_public_key = "..."
If public key isn't passed we will create it for you and store the private key locally under /tmp
Names will be:
/tmp/${prefix}-${cluster_name}-public-key.pub
/tmp/${prefix}-${cluster_name}-private-key.pem
Also we store the keys on key vault as secret: To download keys from key vault use command:
az keyvault secret download --file private.pem --encoding utf-8 --vault-name ${prefix}-${cluster_name} --name private-key --query "value"
az keyvault secret download --file private.pem --encoding utf-8 --vault-name ${prefix}-${cluster_name} --name public-key --query "value"
We support tiering to bucket. In order to setup tiering, you must supply the following variables:
set_obs_integration = true
obs_name = "..."
obs_container_name = "..."
blob_obs_access_key = "..."
In addition, you can supply (and override our default):
tiering_ssd_percent = VALUE
We support creating clients that will be mounted automatically to the cluster.
In order to create clients you need to provide the number of clients you want (by default the number is 0),
for example:
clients_number = 2
This will automatically create 2 clients.
In addition you can supply these optional variables:
client_instance_type = "Standard_D4_v4"
client_nics_num = DESIRED_NUM
In order to mount clients in udp mode you should pass the following param (in addition to the above):
clients_use_dpdk = false
We support creating NFS protocol gateways that will be mounted automatically to the cluster.
In order to create you need to provide the number of protocol gateways instances you want (by default the number is 0),
for example:
nfs_protocol_gateways_number = 1
This will automatically create 2 instances.
In addition you can supply these optional variables:
nfs_protocol_gateway_secondary_ips_per_nic = 3
nfs_protocol_gateway_instance_type = "Standard_D8_v5"
nfs_protocol_gateway_nics_num = 2
nfs_protocol_gateway_disk_size = 48
nfs_protocol_gateway_frontend_cores_num = 1
nfs_setup_protocol = false
In order to create stateless clients, need to set variable:
nfs_setup_protocol = true
We support creating S3 protocol gateways that will be mounted automatically to the cluster.
In order to create you need to provide the number of protocol gateways instances you want (by default the number is 0),
The amount of S3 protocol gateways should be at least 3.
for example:
s3_protocol_gateways_number = 3
This will automatically create 3 instances.
In addition you can supply these optional variables:
s3_protocol_gateway_instance_type = "Standard_D8_v5"
s3_protocol_gateway_nics_num = 2
s3_protocol_gateway_disk_size = 48
s3_protocol_gateway_frontend_cores_num = 1
In order to create stateless clients, need to set variable:
s3_setup_protocol = true
We support creating SMB protocol gateways that will be mounted automatically to the cluster.
In order to create you need to provide the number of protocol gateways instances you want (by default the number is 0),
The amount of SMB protocol gateways should be at least 3.
for example:
smb_protocol_gateways_number = 3
This will automatically create 2 instances.
In addition you can supply these optional variables:
smb_protocol_gateway_secondary_ips_per_nic = 3
smb_protocol_gateway_instance_type = "Standard_D8_v5"
smb_protocol_gateway_nics_num = 2
smb_protocol_gateway_disk_size = 48
smb_protocol_gateway_frontend_cores_num = 1
smb_setup_protocol = false
smb_cluster_name = ""
smb_domain_name = ""
smb_dns_ip_address = ""
In order to create stateless clients, need to set variable:
smb_setup_protocol = true
To join an SMB cluster in Active Directory, need to pass domain username/password,
To join an SMB cluster in Active Directory, need to run manually command:
weka smb domain join <smb_domain_username> <smb_domain_password> [--server smb_server_name]
.
In order to enable SMBW, need to set variable:
smbw_enabled = true
We support weka installation using custom proxy url.
proxy_url = VALUE
Name | Version |
---|---|
terraform | >= 1.4.6 |
azurerm | ~>3.114.0 |
local | ~>2.4.0 |
tls | ~>4.0.4 |
Name | Version |
---|---|
azurerm | ~>3.114.0 |
local | ~>2.4.0 |
tls | ~>4.0.4 |
Name | Source | Version |
---|---|---|
clients | ./modules/clients | n/a |
function_app_subnet_delegation | ./modules/subnet_delegation | n/a |
iam | ./modules/iam | n/a |
logic_app_subnet_delegation | ./modules/subnet_delegation | n/a |
logicapp | ./modules/logic_app | n/a |
network | ./modules/network | n/a |
nfs_protocol_gateways | ./modules/protocol_gateways | n/a |
peering | ./modules/peering_vnets | n/a |
s3_protocol_gateways | ./modules/protocol_gateways | n/a |
smb_protocol_gateways | ./modules/protocol_gateways | n/a |
Name | Description | Type | Default | Required |
---|---|---|---|---|
address_space | The range of IP addresses the virtual network uses. | string |
"10.0.0.0/16" |
no |
allow_ssh_cidrs | Allow port 22, if not provided, i.e leaving the default empty list, the rule will not be included in the SG | list(string) |
[] |
no |
allow_weka_api_cidrs | Allow connection to port 14000 on weka backends from specified CIDRs, by default no CIDRs are allowed. All ports (including 14000) are allowed within Vnet | list(string) |
[] |
no |
application_insights_name | The Application Insights name. | string |
"" |
no |
apt_repo_server | The URL of the apt private repository. | string |
"" |
no |
assign_public_ip | Determines whether to assign public IP to all instances deployed by TF module. Includes backends, clients and protocol gateways. | string |
"auto" |
no |
client_arch | Use arch for ami id, value can be arm64/x86_64. | string |
null |
no |
client_frontend_cores | The client NICs number. | number |
1 |
no |
client_identity_name | The user assigned identity name for the client instances (if empty - new one is created). | string |
"" |
no |
client_instance_type | The client virtual machine type (sku) to deploy. | string |
"" |
no |
client_placement_group_id | The client instances placement group id. Backend placement group can be reused. If not specified placement group will be created automatically | string |
"" |
no |
client_source_image_id | Use weka custom image, ubuntu 20.04 with kernel 5.4 and ofed 5.8-1.1.2.1 / ubuntu arm 20.04 with kernel 5.4 and ofed 5.9-0.5.6.0 | string |
"" |
no |
clients_custom_data | Custom data to pass to the client instances | string |
"" |
no |
clients_number | The number of client virtual machines to deploy. | number |
0 |
no |
clients_use_dpdk | Mount weka clients in DPDK mode | bool |
true |
no |
clients_use_vmss | Use VMSS for clients | bool |
false |
no |
cluster_name | Cluster name | string |
"poc" |
no |
cluster_size | The number of virtual machines to deploy. | number |
6 |
no |
clusterization_target | The clusterization target | number |
null |
no |
containers_config_map | Maps the number of objects and memory size per machine type. | map(object({ |
{ |
no |
create_lb | Create backend and UI load balancers for weka cluster. | bool |
true |
no |
create_nat_gateway | NAT needs to be created when no public ip is assigned to the backend, to allow internet access | bool |
false |
no |
create_storage_account_private_links | Create private links for storage accounts (needed in case if public network access for the storage account is disabled). | bool |
false |
no |
debug_down_backends_removal_timeout | Don't change this value without consulting weka support team. Timeout for removing down backends. Valid time units are ns, us (or µs), ms, s, m, h. | string |
"3h" |
no |
default_disk_size | The default disk size. | number |
48 |
no |
deployment_container_name | Name of exising deployment container | string |
"" |
no |
deployment_file_share_name | Name of exising deployment file share. Will use '<deployment_storage_account_name>-share' name if not provided. | string |
"" |
no |
deployment_function_app_code_blob | The path to the function app code blob file. | string |
"" |
no |
deployment_storage_account_name | Name of exising deployment storage account | string |
"" |
no |
enable_application_insights | Enable Application Insights. | bool |
true |
no |
function_access_restriction_enabled | Allow public access, Access restrictions apply to inbound access to internal vent | bool |
false |
no |
function_app_dist | Function app code dist | string |
"release" |
no |
function_app_identity_name | The user assigned identity name for the function app (if empty - new one is created). | string |
"" |
no |
function_app_log_level | Log level for function app (from -1 to 5). See https://github.com/rs/zerolog#leveled-logging | number |
1 |
no |
function_app_storage_account_container_prefix | Weka storage account container name prefix | string |
"weka-tf-functions-deployment-" |
no |
function_app_storage_account_prefix | Weka storage account name prefix | string |
"weka" |
no |
function_app_subnet_delegation_cidr | Subnet delegation enables you to designate a specific subnet for an Azure PaaS service. | string |
"10.0.1.0/25" |
no |
function_app_subnet_delegation_id | Required to specify if subnet_name were used to specify pre-defined subnets for weka. Function subnet delegation requires an additional subnet, and in the case of pre-defined networking this one also should be pre-defined | string |
"" |
no |
function_app_version | Function app code version (hash) | string |
"f574113c680e317e7bca869159e0d168" |
no |
get_weka_io_token | The token to download the Weka release from get.weka.io. | string |
"" |
no |
hotspare | Number of hotspares to set on weka cluster. Refer to https://docs.weka.io/overview/ssd-capacity-management#hot-spare | number |
1 |
no |
install_cluster_dpdk | Install weka cluster with DPDK | bool |
true |
no |
install_weka_url | The URL of the Weka release download tar file. | string |
"" |
no |
instance_type | The virtual machine type (sku) to deploy. | string |
"Standard_L8s_v3" |
no |
key_vault_purge_protection_enabled | Enable purge protection for the key vault. | bool |
false |
no |
log_analytics_workspace_id | The Log Analytics workspace id. | string |
"" |
no |
logic_app_identity_name | The user assigned identity name for the logic app (if empty - new one is created). | string |
"" |
no |
logic_app_subnet_delegation_cidr | Subnet delegation enables you to designate a specific subnet for an Azure PaaS service. | string |
"10.0.3.0/25" |
no |
logic_app_subnet_delegation_id | Required to specify if subnet_name were used to specify pre-defined subnets for weka. Logicapp subnet delegation requires an additional subnet, and in the case of pre-defined networking this one also should be pre-defined | string |
"" |
no |
nfs_deployment_container_name | Name of exising protocol deployment container | string |
"" |
no |
nfs_interface_group_name | Interface group name. | string |
"weka-ig" |
no |
nfs_protocol_gateway_disk_size | The protocol gateways' default disk size. | number |
48 |
no |
nfs_protocol_gateway_fe_cores_num | The number of frontend cores on single protocol gateway machine. | number |
1 |
no |
nfs_protocol_gateway_instance_type | The protocol gateways' virtual machine type (sku) to deploy. | string |
"Standard_D8_v5" |
no |
nfs_protocol_gateway_secondary_ips_per_nic | Number of secondary IPs per single NIC per protocol gateway virtual machine. | number |
0 |
no |
nfs_protocol_gateways_number | The number of protocol gateway virtual machines to deploy. | number |
0 |
no |
nfs_setup_protocol | Config protocol, default if false | bool |
false |
no |
placement_group_id | Proximity placement group to use for the vmss. If not passed, will be created automatically. | string |
"" |
no |
prefix | Prefix for all resources | string |
"weka" |
no |
private_dns_rg_name | The private DNS zone resource group name. Required when private_dns_zone_name is set. | string |
"" |
no |
private_dns_zone_name | The private DNS zone name. | string |
"" |
no |
private_dns_zone_use | Determines whether to use private DNS zone. Required for LB record creation. | bool |
true |
no |
protection_level | Cluster data protection level. | number |
2 |
no |
protocol_gateways_identity_name | The user assigned identity name for the protocol gateways instances (if empty - new one is created). | string |
"" |
no |
proxy_url | Weka home proxy url | string |
"" |
no |
read_function_zip_from_storage_account | Read function app zip from storage account (is read from public distribution storage account by default). | bool |
false |
no |
rg_name | A predefined resource group in the Azure subscription. | string |
n/a | yes |
s3_protocol_gateway_disk_size | The protocol gateways' default disk size. | number |
48 |
no |
s3_protocol_gateway_fe_cores_num | The number of frontend cores on single protocol gateway machine. | number |
1 |
no |
s3_protocol_gateway_instance_type | The protocol gateways' virtual machine type (sku) to deploy. | string |
"Standard_D8_v5" |
no |
s3_protocol_gateways_number | The number of protocol gateway virtual machines to deploy. | number |
0 |
no |
s3_setup_protocol | Config protocol, default if false | bool |
false |
no |
script_post_cluster_creation | Script to run after cluster creation | string |
"" |
no |
script_pre_start_io | Script to run before starting IO | string |
"" |
no |
set_dedicated_fe_container | Create cluster with FE containers | bool |
false |
no |
sg_id | The security group id. | string |
"" |
no |
smb_cluster_name | The name of the SMB setup. | string |
"Weka-SMB" |
no |
smb_create_private_dns_resolver | Create dns resolver for smb with outbound rule | bool |
false |
no |
smb_dns_ip_address | DNS IP address | string |
"" |
no |
smb_dns_resolver_subnet_delegation_cidr | Cidr of dns resolver of subnet, for SMB | string |
"10.0.4.0/28" |
no |
smb_dns_resolver_subnet_delegation_id | Required to specify if subnet_id were used to specify pre-defined for SMB dns resolver subnet, requires an additional subnet, '/subscriptions/../resourceGroups/../providers/Microsoft.Network/virtualNetworks/../subnets/..' | string |
"" |
no |
smb_domain_name | The domain to join the SMB cluster to. | string |
"" |
no |
smb_protocol_gateway_disk_size | The protocol gateways' default disk size. | number |
48 |
no |
smb_protocol_gateway_fe_cores_num | The number of frontend cores on single protocol gateway machine. | number |
1 |
no |
smb_protocol_gateway_instance_type | The protocol gateways' virtual machine type (sku) to deploy. | string |
"Standard_D8_v5" |
no |
smb_protocol_gateway_secondary_ips_per_nic | Number of secondary IPs per single NIC per protocol gateway virtual machine. | number |
0 |
no |
smb_protocol_gateways_number | The number of protocol gateway virtual machines to deploy. | number |
0 |
no |
smb_setup_protocol | Config protocol, default if false | bool |
false |
no |
smbw_enabled | Enable SMBW protocol. This option should be provided before cluster is created to leave extra capacity for SMBW setup. | bool |
true |
no |
source_image_id | Use weka custom image, ubuntu 20.04 with kernel 5.4 and ofed 5.8-1.1.2.1 | string |
"/communityGalleries/WekaIO-ddbef83d-dec1-42d0-998a-3c083f1450b7/images/weka_custom_image/versions/1.0.1" |
no |
ssh_public_key | Ssh public key to pass to vms. | string |
null |
no |
storage_account_allowed_ips | IP ranges to allow access from the internet or your on-premises networks to storage accounts. | list(string) |
[] |
no |
storage_account_public_network_access | Public network access to the storage accounts. | string |
"Enabled" |
no |
storage_blob_private_dns_zone_name | The private DNS zone name for the storage account (blob). | string |
"privatelink.blob.core.windows.net" |
no |
stripe_width | Stripe width = cluster_size - protection_level - 1 (by default). | number |
-1 |
no |
subnet_name | The subnet name. | string |
"" |
no |
subnet_prefix | Address prefixes to use for the subnet | string |
"10.0.2.0/24" |
no |
subscription_id | The subscription id for the deployment. | string |
n/a | yes |
tags_map | A map of tags to assign the same metadata to all resources in the environment. Format: key:value. | map(string) |
{} |
no |
tiering_blob_obs_access_key | The access key of the existing Blob object store container. | string |
"" |
no |
tiering_enable_obs_integration | Determines whether to enable object stores integration with the Weka cluster. Set true to enable the integration. | bool |
false |
no |
tiering_enable_ssd_percent | When set_obs_integration is true, this variable sets the capacity percentage of the filesystem that resides on SSD. For example, for an SSD with a total capacity of 20GB, and the tiering_ssd_percent is set to 20, the total available capacity is 100GB. | number |
20 |
no |
tiering_obs_container_name | Name of existing obs conatiner name | string |
"" |
no |
tiering_obs_name | Name of existing obs storage account | string |
"" |
no |
tiering_obs_start_demote | Target tiering cue (in seconds) before starting upload data to OBS (turning it into read cache). Default is 10 seconds. | number |
10 |
no |
tiering_obs_target_ssd_retention | Target retention period (in seconds) before tiering to OBS (how long data will stay in SSD). Default is 86400 seconds (24 hours). | number |
86400 |
no |
traces_per_ionode | The number of traces per ionode. Traces are low-level events generated by Weka processes and are used as troubleshooting information for support purposes. | number |
10 |
no |
user_data | User data to pass to vms. | string |
"" |
no |
vm_username | Provided as part of output for automated use of terraform, in case of custom AMI and automated use of outputs replace this with user that should be used for ssh connection | string |
"weka" |
no |
vmss_identity_name | The user assigned identity name for the vmss instances (if empty - new one is created). | string |
"" |
no |
vmss_single_placement_group | Sets single_placement_group option for vmss. If true, a scale set is composed of a single placement group, and has a range of 0-100 VMs. | bool |
true |
no |
vnet_name | The virtual network name. | string |
"" |
no |
vnet_rg_name | Resource group name of vnet. Will be used when vnet_name is not provided. | string |
"" |
no |
vnets_to_peer_to_deployment_vnet | List of vent-name:resource-group-name to peer | list(object({ |
[] |
no |
weka_home_url | Weka Home url | string |
"" |
no |
weka_tar_storage_account_id | ### private blob | string |
"" |
no |
weka_version | The Weka version to deploy. | string |
"" |
no |
zone | The zone in which the resources should be created. | string |
null |
no |
Name | Description |
---|---|
backend_ips | If 'assign_public_ip' is set to true, it will output the public ips, If no it will output the private ips |
backend_lb_private_ip | Backend load balancer ip address |
client_ips | If 'private_network' is set to false, it will output clients public ips, otherwise private ips. |
client_vmss_ips | If 'private_network' is set to false, it will output clients public ips, otherwise private ips. |
clients_vmss_name | n/a |
cluster_helper_commands | Useful commands and script to interact with weka cluster |
function_app_name | Function app name |
function_key_name | Function app key name |
functions_url | Functions url and body for api request |
key_vault_name | Keyault name |
nfs_vmss_name | NFS protocol gateway vmss name |
ppg_id | Placement proximity group id |
private_ssh_key | If 'ssh_public_key' is set to null and no file provided, it will output the private ssh key location. |
s3_protocol_gateway_ips | If 'private_network' is set to false, it will output smb protocol gateway public ips, otherwise private ips. |
sg_id | Security group id |
smb_protocol_gateway_ips | If 'private_network' is set to false, it will output smb protocol gateway public ips, otherwise private ips. |
subnet_name | Subnet name |
vm_username | Provided as part of output for automated use of terraform, ssh user to weka cluster vm |
vmss_name | n/a |
vnet_name | Virtual network name |
vnet_rg_name | Virtual network resource group name |
weka_cluster_admin_password_secret_name | Weka cluster admin password secret name |