3301-3350 of 8586 results (15ms)
2021-04-07 §
21:22 <mutante> mw2397 through mw2402 - pooled as new API appservers after scap pull and all monitoring green (T278396) [production]
21:05 <mutante> mw2397 - mw2402 - scap pull [production]
2021-04-06 §
18:41 <dduvall@deploy1002> Finished scap: testwikis wikis to 1.36.0-wmf.38 (duration: 33m 31s) [production]
18:10 <dduvall@deploy1002> Started scap: testwikis wikis to 1.36.0-wmf.38 [production]
2021-04-02 §
21:07 <mutante> mw2383 through mw2394 - 'uptime && scap pull' via ssh -C (not cumin because it needs to run as non-root) [production]
20:58 <mutante> mw238* - scap pull via cumin not possible because it doesnt work as root [production]
2021-04-01 §
20:01 <mutante> mw2379, mw2380, mw2381, mw2382 - scap pull [production]
19:34 <twentyafterfour@deploy1002> scap sync-l10n completed (1.36.0-wmf.37) (duration: 02m 38s) [production]
2021-03-30 §
21:02 <legoktm> scap pulling on mw1298 [production]
20:33 <twentyafterfour@deploy1002> Finished scap: testwikis wikis to 1.36.0-wmf.37 refs T278343 (duration: 80m 32s) [production]
20:02 <twentyafterfour> when syncing 1.36.0-wmf.37 promote to testwikis, one server failed: server mw1298.eqiad.wmnet and two more appear to be hung because scap is stuck at 2 left 99% without making any progress for a long time now. refs T278343 [production]
19:13 <twentyafterfour@deploy1002> Started scap: testwikis wikis to 1.36.0-wmf.37 refs T278343 [production]
2021-03-26 §
17:42 <hashar@deploy1002> Finished scap: Revert "Add change tags for media additions/removals" - T266067 T278429 (duration: 31m 43s) [production]
17:10 <hashar@deploy1002> Started scap: Revert "Add change tags for media additions/removals" - T266067 T278429 [production]
2021-03-25 §
18:09 <marxarelli> scap sync-file .pipeline Config: [[gerrit:674132|Include patches in restricted image (T271274)]] [production]
00:34 <mutante> mw2377, mw2378 - first scap pull [production]
2021-03-23 §
13:55 <hashar@deploy1002> Finished scap: Promote testwikis from 1.36.0-wmf.35 to 1.36.0-wmf.36 - T274940 (duration: 31m 57s) [production]
13:23 <hashar@deploy1002> Started scap: Promote testwikis from 1.36.0-wmf.35 to 1.36.0-wmf.36 - T274940 [production]
10:02 <hashar> scap clean --delete 1.36.0-wmf.32 # T274940 [production]
09:53 <hashar> scap prep 1.36.0-wmf.36 # T274940 [production]
2021-03-19 §
19:24 <mutante> deploy2002 - re-enabled puppet, reverted patch of scap-sync-master [production]
18:46 <mutante> deploy2002 - disable puppet, copy modified version of scap-master-sync over it that does not --exclude="**/cache/l10n/*.cdb" (for T275826) [production]
2021-03-16 §
19:31 <dancy@deploy1002> Finished scap: testwikis wikis to 1.36.0-wmf.35 (duration: 33m 41s) [production]
18:58 <dancy@deploy1002> Started scap: testwikis wikis to 1.36.0-wmf.35 [production]
2021-03-09 §
18:54 <brennen@deploy1002> Finished scap: testwikis wikis to 1.36.0-wmf.34 (duration: 47m 25s) [production]
18:12 <brennen@deploy1002> Started scap: testwikis wikis to 1.36.0-wmf.34 [production]
2021-03-02 §
09:46 <liw@deploy1002> Finished scap: testwikis wikis to 1.36.0-wmf.33 (duration: 36m 20s) [production]
09:12 <liw@deploy1002> Started scap: testwikis wikis to 1.36.0-wmf.33 [production]
2021-03-01 §
22:12 <twentyafterfour@deploy1002> Finished scap: (no justification provided) (duration: 16m 24s) [production]
21:57 <twentyafterfour> running scap sync from the new server deply1002 [production]
21:56 <twentyafterfour@deploy1002> Started scap: (no justification provided) [production]
21:49 <mutante> deploy1002 - removed scap-global-lock, unlocked scap [production]
21:18 <mutante> mw1262 - running puppet to switch to new deployment server, scap pull [production]
21:05 <mutante> re-enabling puppet on deploy1001 - running puppet on deploy*, switching eqiad scap master and deployment_server globally (T265963) [production]
20:37 <mutante> deploy1001 - disable puppet and manually create scap-global-lock - NO DEPLOYMENTS [production]
2021-02-25 §
23:55 <mutante> deploy1002, deploy2002 - scap-master-sync deploy1001.eqiad.wmnet (T265963) [production]
23:39 <mutante> deploy2001 - scap-master-sync from deploy1001 runs and attempts to --delete files to stay in sync but fails to do so because *.cdb files are in cache dirs and rsync does not want to delete non-empty directories, this leads to build up of the size of /srv/mediawiki-staging to 10 times the size of eqiad [production]
23:34 <mutante> deploy2001 - scap-master-sync from deploy1001 [production]
23:13 <mutante> deploy1002 - /usr/local/bin/scap-master-sync deploy1001.eqiad.wmnet [production]
2021-02-24 §
09:06 <volans> run "sudo find . -user root -exec chown netbox. '{}' \;" in /srv/deployment/netbox/deploy-cache/revs on netbox* hosts to prevent scap failures on cleanup - T265084 [production]
2021-02-23 §
21:28 <jhuneidi@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.32 refs T274936 (duration: 36m 52s) [production]
20:52 <jhuneidi@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.32 refs T274936 [production]
2021-02-22 §
16:51 <Urbanecm> Run scap pull on mwmaint1002 to clear any local changes [production]
2021-02-19 §
20:33 <mutante> mw1261, mw1270 - scap pull [production]
20:32 <mutante> mw1287 - scap pull [production]
01:22 <mutante> mwmaint2001 back on buster and back in scap dsh groups (if anything pops up you can revert 665175) [production]
00:54 <mutante> mwmaint2001 - back from reimage - scap pull [production]
2021-02-16 §
20:20 <mutante> mwdebug1002 has been recreated on buster and has been repooled after scap pull - you can find a .tar.gz in your home with the contents of your home before reimaging, fingerprint at T274023#6835116 [production]
18:49 <dduvall@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.31 (duration: 49m 37s) [production]
18:04 <dduvall@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.31 [production]