close
Page MenuHomePhabricator

Cloud-VPS (Project-requests)Milestone
ActivePublic

Details

Description

Requests for creating or deleting Cloud-VPS projects. See openstack-browser for a list of existing projects and their members. For quota changes see Cloud-VPS (Quota-requests)

Before you request a project

  1. Confirm you cannot run your code/bot/thing on Toolforge. You can request access to Toolforge by following the instructions on wikitech. With Toolforge, the underlying infrastructure will be taken care of for you, and you can focus on writing your tool / bot / researchy-thing.
  1. Only request a VPS project if you need root on the VM for a specific purpose (or need testing VMs for MediaWiki changes, etc). Remember that if you have root you have responsibilities too! :)

Maintainer responsibilities

Each Cloud VPS project must have one or more active maintainers. Maintainers must have an active Phabricator account, must subscribe to the cloud-announce mailing list, and must respond to tasks, emails, and requests for action on the part of WMCS staff and admins.

In addition, maintainers of VMs in Cloud VPS projects must provide basic maintenance on their servers: puppet needs to be kept in working order, upgrades will need to be applied periodically, and VMs will need to be periodically deleted and recreated with updated OS images.

If a project is not actively maintained, or WMCS staff are unable to contact maintainers for a long period of time, that project will be shut down and ultimately deleted along with all associated configuration and data.

Project scope

Cloud VPS projects should be scoped based around concrete products or software projects, rather than the team working on them. The three main problems that we (the Cloud Services team) have seen in the past with team ownership/scope for Cloud VPS projects are:

  • Team gets disbanded/reorganized but its project needs to live on due to hosting of important VMs
  • Difficulty establishing who is the primary point of contact for a given VM when trying to reclaim quota or fix a broken instance
  • Tendency to close membership/participation to only team members rather than inviting participation by other volunteers

There are things that can be done to mitigate these problems, but the easiest thing to do is to create more targeted projects that are scoped to a product/project rather than a team. This can become a burden in other ways if a common group of developers is active on a large number of such projects, so we are willing to be flexible if good cause can be shown for project consolidation.

For more guidance, see https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS_project

Requesting a new project

To request a new project, create a task using this link, and fill out the requested information by <replacing the instructions>

Note that there's currently some restrictions on the name of the project: it should not contain underscores or special characters, because it will be used in DNS records.

Process

Requests can be granted when approved by a quorum of at least two Cloud VPS administrators. If a decision is not made asynchronously, requests are processed by the cloud-services-team during the Cloud Services team meeting every Thursday (8:00 PST) that the meeting is held. This schedule may be effected by holidays, conferences, or other unavailability.

Once approved, creation of the project in OpenStack will be handled by a member of the Cloud VPS admin team. This will typically happen within 1-2 days, but may take longer due to availability of computing resources or higher priority administration tasks.

See also

Recent Activity

Mon, Jul 13

Aklapper placed T431996: Request creation of vps up for grabs.

Project Name: wikidata

Mon, Jul 13, 1:36 PM · Cloud-VPS (Project-requests)
Chaosnb created T431996: Request creation of vps.
Mon, Jul 13, 12:21 PM · Cloud-VPS (Project-requests)

Wed, Jul 8

gmodena added a comment to T431531: Request creation of wdp-gitlab-runner VPS project.

Terrific! Thank you so much for picking this task up so quickly. Now let's build :)

Wed, Jul 8, 12:37 PM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
dcaro closed T431531: Request creation of wdp-gitlab-runner VPS project as Resolved.
Wed, Jul 8, 12:31 PM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
dcaro moved T431531: Request creation of wdp-gitlab-runner VPS project from Doing to Done on the User-dcaro board.
Wed, Jul 8, 12:31 PM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
dcaro added a comment to T431531: Request creation of wdp-gitlab-runner VPS project.

Done, enjoy :)

Wed, Jul 8, 12:31 PM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
Maintenance_bot removed a project from T431531: Request creation of wdp-gitlab-runner VPS project: Patch-For-Review.
Wed, Jul 8, 12:31 PM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
CodeReviewBot added a comment to T431531: Request creation of wdp-gitlab-runner VPS project.

projects: added project wdp-gitlab-runner

Wed, Jul 8, 12:23 PM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
dcaro changed the status of T431531: Request creation of wdp-gitlab-runner VPS project from Open to In Progress.
Wed, Jul 8, 12:22 PM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
dcaro moved T431531: Request creation of wdp-gitlab-runner VPS project from To refine to Doing on the User-dcaro board.
Wed, Jul 8, 12:22 PM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
CodeReviewBot added a project to T431531: Request creation of wdp-gitlab-runner VPS project: Patch-For-Review.

group_199_bot_f98be072172e323ae6d1441939d3e461 opened https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/320

Wed, Jul 8, 12:14 PM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
dcaro claimed T431531: Request creation of wdp-gitlab-runner VPS project.
Wed, Jul 8, 10:10 AM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
fgiunchedi added a comment to T431531: Request creation of wdp-gitlab-runner VPS project.

+1 LGTM

Wed, Jul 8, 10:05 AM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
gmodena added a comment to T431531: Request creation of wdp-gitlab-runner VPS project.

👍 just checking :)

Wed, Jul 8, 9:48 AM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
dcaro added a comment to T431531: Request creation of wdp-gitlab-runner VPS project.

Hi @dcaro !

AFAIK those are community driven hosting projects that run their own query service.

Wikidata Platform we would need something dedicated to our team specifically to build the QLever database (and possibly other compute heavy builds in the future).

Community projects can use the artifacts from our build if desired (we will publish as a docker image in the the wikimedia registry), but I would prefer to keep our use case and VM orthogonal to existing projects.

Wed, Jul 8, 9:42 AM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
gmodena added a comment to T431531: Request creation of wdp-gitlab-runner VPS project.

AFAIK those are community driven hosting projects that run their own query service.

Wed, Jul 8, 8:54 AM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
dcaro added a comment to T431531: Request creation of wdp-gitlab-runner VPS project.

Hi! Given that it's one VM, and might be closely related, is this something that can be part of the query-service project? Or wikiqlever?

Wed, Jul 8, 8:31 AM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
gmodena added a project to T431531: Request creation of wdp-gitlab-runner VPS project: Wikidata Platform Team.
Wed, Jul 8, 8:31 AM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
gmodena renamed T431531: Request creation of wdp-gitlab-runner VPS project from Request creation of wdqs-qlever-build-runner VPS project to Request creation of wdp-gitlab-runner VPS project.
Wed, Jul 8, 8:31 AM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)
gmodena created T431531: Request creation of wdp-gitlab-runner VPS project.
Wed, Jul 8, 8:23 AM · User-dcaro, Wikidata Platform Team, Cloud-VPS (Project-requests)

Mon, Jul 6

isarantopoulos moved T425909: Request creation of wikitts VPS project from Q1 FY2026-27 to Task Archive on the Machine-Learning-Team board.
Mon, Jul 6, 7:57 AM · Machine-Learning-Team, Cloud-VPS (Project-requests)

Fri, Jul 3

dschwen added a comment to T429446: Check current usage of project maps on CloudVPS.

Thank you!

Fri, Jul 3, 2:42 AM · Maps, Cloud-VPS (Project-requests)

Thu, Jul 2

bd808 updated the task description for T429446: Check current usage of project maps on CloudVPS.
Thu, Jul 2, 10:19 PM · Maps, Cloud-VPS (Project-requests)

Tue, Jun 30

fnegri closed T429446: Check current usage of project maps on CloudVPS as Declined.

Closing as "Declined", there are still active users of this project as described above.

Tue, Jun 30, 4:30 PM · Maps, Cloud-VPS (Project-requests)
bd808 added a comment to T429446: Check current usage of project maps on CloudVPS.

Discussing with SRE I/F it came out that probably this project is not needed anymore and can be deleted.
Opening the task for the papertrail and to get an official confirmation from @elukey and @Jgiannelos.

Tue, Jun 30, 3:22 PM · Maps, Cloud-VPS (Project-requests)

Thu, Jun 25

JJMC89 changed the status of T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project from Resolved to Declined.
Thu, Jun 25, 4:12 PM · Cloud-VPS (Project-requests)
fgiunchedi closed T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project as Resolved.

I am tentatively resolving this task since:

Thu, Jun 25, 4:10 PM · Cloud-VPS (Project-requests)

Wed, Jun 24

Bugreporter added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.
Wed, Jun 24, 6:12 PM · Cloud-VPS (Project-requests)
Pppery added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

You need someone in https://meta.wikimedia.org/wiki/Special:ListUsers?username=&group=oauthadmin&wpsubmit=&wpFormIdentifier=mw-listusers-form&limit=50 to do it.

Wed, Jun 24, 2:33 PM · Cloud-VPS (Project-requests)
Aklapper added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Yeah, another ticket makes most sense (I don't know the workflow or the required group membership from the top of my head). Thanks!

Wed, Jun 24, 10:36 AM · Cloud-VPS (Project-requests)
fgiunchedi added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Ok, thanks, I never used Toolforge - according to the instruction - I sent the Toolforge membership request.

Wed, Jun 24, 9:23 AM · Cloud-VPS (Project-requests)

Tue, Jun 23

vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Created this oauth client without "allow consumer to cpecify a callback" - its a mistake, already did a new one.

Tue, Jun 23, 4:51 PM · Cloud-VPS (Project-requests)
vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Ok, thanks, I never used Toolforge - according to the instruction - I sent the Toolforge membership request.

Tue, Jun 23, 4:00 PM · Cloud-VPS (Project-requests)
fgiunchedi added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

And does Toolforge accept zip/rar that are a few GB in size?

Tue, Jun 23, 3:46 PM · Cloud-VPS (Project-requests)
fgiunchedi added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Ok, if I will be able to install official Telegram server to Toolforge - I can try it. I will have ssh to that Toolforge, right?

Will implement oauth tokens now...

Tue, Jun 23, 3:43 PM · Cloud-VPS (Project-requests)
vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

And does Toolforge accept zip/rar that are a few GB in size?

Tue, Jun 23, 3:43 PM · Cloud-VPS (Project-requests)
vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Ok, if I will be able to install official Telegram server to Toolforge - I can try it. I will have ssh to that Toolforge, right?

Tue, Jun 23, 3:40 PM · Cloud-VPS (Project-requests)
fgiunchedi added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

this sounds to me like a great use case for a Toolforge tool.

For this project, we need to host the official Telegram server https://github.com/tdlib/telegram-bot-api for long pooling - is it possible?

Tue, Jun 23, 3:33 PM · Cloud-VPS (Project-requests)
vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

I can implement oauth token, if you think that it will be better.

Tue, Jun 23, 3:27 PM · Cloud-VPS (Project-requests)
vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

this sounds to me like a great use case for a Toolforge tool.

Tue, Jun 23, 3:25 PM · Cloud-VPS (Project-requests)
fgiunchedi added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Thank you for explaining the workflow -- this sounds to me like a great use case for a Toolforge tool. Have you considered deploying on Toolforge instead?

Tue, Jun 23, 3:14 PM · Cloud-VPS (Project-requests)

Mon, Jun 22

vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

...decryption happens only in memory, per upload.

Mon, Jun 22, 6:23 PM · Cloud-VPS (Project-requests)
vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

...and bot deletes the Telegram message with the password.

Mon, Jun 22, 6:22 PM · Cloud-VPS (Project-requests)
vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Bot asks for username and password. We store encrypted passwords (AES-256-GCM) in sqlite.

Mon, Jun 22, 6:22 PM · Cloud-VPS (Project-requests)
Aklapper added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

As "user bot passwords" were mentioned, what is the authorization workflow between a user using this bot and the uploader account of the file on Commons?

Mon, Jun 22, 5:44 PM · Cloud-VPS (Project-requests)
vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Data flow: user sends photo(s) (even zip/rar are supported) - Telegram webhook routes them to this VPS - this VPS uploads photo(s) to Wikimedia Commons (with correct wikitext), optionally returning links to uploaded files.

Mon, Jun 22, 4:20 PM · Cloud-VPS (Project-requests)
fgiunchedi moved T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project from Inbox to Feedback needed on the Cloud-VPS (Project-requests) board.
Mon, Jun 22, 11:23 AM · Cloud-VPS (Project-requests)
fgiunchedi added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Thank you for reaching out and the links; I'm not familiar with the telegram bot api, how would the data flow look like in case of uploads? would you mind explaning where data is going to flow from/to and be stored ? thank you

Mon, Jun 22, 10:54 AM · Cloud-VPS (Project-requests)

Sat, Jun 20

vitaly-zdanevich added a comment to T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.

Oops for files uploading from a user to public Telgram API - the limit is 20 MB https://core.telegram.org/bots/api#file

Sat, Jun 20, 5:27 PM · Cloud-VPS (Project-requests)
vitaly-zdanevich created T429729: Request creation of wikimedia_commons_uploader_telegram_bot VPS project.
Sat, Jun 20, 5:25 PM · Cloud-VPS (Project-requests)