1801-1850 of 10000 results (67ms)
2019-11-19 ยง
23:42 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=phab2001-vcs.codfw.wmnet [production]
21:45 <XioNoX> rebooting pfw3-codfw:node1 for upgrade - T235150 [production]
21:14 <XioNoX> rebooting pfw3-codfw for upgrade - T235150 [production]
20:59 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
20:17 <gehel> completed reloading data from wdqs1007 to wdqs1004 - after failed test of merging updater - T212826 [production]
20:14 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
20:10 <XioNoX> homer push on mgmt routers [production]
20:09 <mutante> phab1003 after merging gerrit:551910 puppet now also stopped the actual aphlict service and removed the systemd unit file. had to manually run 'systemctl reset-failed' though to clean systemd status and avoid icinga alert (T238593) [production]
20:07 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
19:20 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
19:18 <dzahn@cumin1001> END (ERROR) - Cookbook sre.ganeti.makevm (exit_code=97) [production]
19:18 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
19:08 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@6e6bd42]: Prevent expensive content transforms from blocking the event loop (T229286) (duration: 06m 49s) [production]
19:01 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@6e6bd42]: Prevent expensive content transforms from blocking the event loop (T229286) [production]
19:00 <elukey> regenerate TLS cert for yarn.wikimedia.org (containing SANs for all analytics UIs) to add datasets.w.o SAN (site was failing due to ATS not being able to contact thorium) [production]
18:59 <rlazarus> restarted php7.2-fpm on wtp2001, wtp2002 [production]
18:56 <rlazarus> restarted php7.2-fpm on wtp1025, wtp1026 [production]
18:35 <catrope@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/VisualEditor/: Unbreak instrumentation of init events (duration: 00m 53s) [production]
18:34 <ssastry@deploy1001> Finished deploy [parsoid/deploy@6e7cffd]: Updating Parsoid to 1a1105a7 (duration: 02m 04s) [production]
18:32 <ssastry@deploy1001> Started deploy [parsoid/deploy@6e7cffd]: Updating Parsoid to 1a1105a7 [production]
18:30 <mutante> icinga config - manually added team-dcops, started icinga [production]
18:20 <addshore@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/Wikidata.org: T221774 - Wikidata.org extension (queryservice maxlag, hook) [[gerrit:551858]] (duration: 00m 53s) [production]
18:12 <RoanKattouw> That was eowiktionary, not eowikisource [production]
18:09 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Configure default search namespaces for eowikisource (T237792) (duration: 00m 52s) [production]
17:43 <addshore@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/Wikidata.org: T221774 - Wikidata.org extension (queryservice maxlag, maint script) [[gerrit:551857]] (duration: 00m 52s) [production]
17:39 <gehel@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) [production]
17:11 <addshore@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/Wikidata.org: T221774 - Wikidata.org extension (queryservice maxlag) [[gerrit:551855]] [[gerrit:551856]] (duration: 00m 54s) [production]
17:02 <volker-e@deploy1001> Finished deploy [design/style-guide@d73818a]: Deploy design/style-guide: (duration: 00m 07s) [production]
17:02 <volker-e@deploy1001> Started deploy [design/style-guide@d73818a]: Deploy design/style-guide: [production]
16:58 <ema> pool cp2007 with ATS backend T227432 [production]
16:30 <ema@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:28 <ema@cumin2001> START - Cookbook sre.hosts.downtime [production]
16:25 <moritzm> installing glib2.0 security updates [production]
16:21 <mutante> phab1003 - puppet restarts aphlict service even with "phabricator_aphlict_enabled: false" in Hiera. But it does properly remove the proxy config lines from apache. so service is running but not used. (T238593) [production]
16:17 <mutante> phab1003 - systemctl stop aphlict (proxy config in apache is disabled as well as disabled in ATS) (T238593) [production]
16:15 <gehel> reloading data from wdqs1007 to wdqs1004 - after failed test of merging updater - T212826 [production]
16:14 <gehel@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
16:10 <ema> depool cp2007 and reimage as text_ats T227432 [production]
16:09 <ema> pool cp2006 with ATS backend T227432 [production]
15:59 <mobrovac@deploy1001> Finished deploy [restbase/deploy@564b2c6]: New Parsoid/PHP config structure (duration: 02m 11s) [production]
15:57 <mobrovac@deploy1001> Started deploy [restbase/deploy@564b2c6]: New Parsoid/PHP config structure [production]
15:37 <ema@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:34 <ema@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:27 <mobrovac@deploy1001> Finished deploy [restbase/deploy@5e7f759]: Switch test.wp and test2.wp to Parsoid/PHP - T229015 (duration: 14m 22s) [production]
15:15 <ema> depool cp2006 and reimage as text_ats T227432 [production]
15:13 <mobrovac@deploy1001> Started deploy [restbase/deploy@5e7f759]: Switch test.wp and test2.wp to Parsoid/PHP - T229015 [production]
15:09 <mobrovac@deploy1001> Finished deploy [restbase/deploy@5e7f759] (dev-cluster): Switch test.wp and test2.wp to Parsoid/PHP (duration: 02m 58s) [production]
15:07 <ema> pool cp2004 with ATS backend T227432 [production]
15:06 <mobrovac@deploy1001> Started deploy [restbase/deploy@5e7f759] (dev-cluster): Switch test.wp and test2.wp to Parsoid/PHP [production]
14:38 <ema@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]