2351-2400 of 10000 results (58ms)
2018-12-06 §
12:55 <moritzm> installing nginx updates on mw in codfw [production]
11:59 <moritzm> installing nginx updates on mw canaries [production]
10:59 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: T85757: repool db1096:3315 (duration: 00m 47s) [production]
10:56 <volans> disable event handler on Icinga for ms-be2047 MD Raid and MegaRAID checks, it's spamming Phabricator - T209921 [production]
10:56 <banyek> repooling db1096 for schema change - T85757 [production]
10:36 <banyek@deploy1001> Synchronized wmf-config/db-eqiad.php: T85757: depool db1096:3315 (duration: 00m 49s) [production]
10:29 <banyek> depooling db1096 for schema change - T85757 [production]
09:56 <dcausse> elastic@codfw cleanup: deleting wikidatawiki_content_1537469318 index (failed reindex probably) [production]
01:03 <tstarling@deploy1001> Synchronized w/fatal-error.php: (no justification provided) (duration: 00m 46s) [production]
00:55 <tstarling@deploy1001> Synchronized w/fatal-error.php: (no justification provided) (duration: 00m 47s) [production]
00:42 <tstarling@deploy1001> Synchronized w/fatal-error.php: (no justification provided) (duration: 00m 46s) [production]
00:40 <tstarling@deploy1001> Synchronized private/FatalErrorSettings.php: (no justification provided) (duration: 00m 46s) [production]
00:38 <tstarling@deploy1001> Synchronized private/FatalErrorSettings.php: (no justification provided) (duration: 00m 46s) [production]
00:15 <mutante> MPM prefork tweaks for high load systems are applied again (apparently they were not since a change in the past that resulted in 2 competing configs in mods-enabled and conf-enabled with the latter one being loaded last and containing the package defaults [production]
00:13 <mutante> re-enabling puppet on phabricator, applying change that adds php-fpm support on stretch ..which doesnt affect phab1001 (prod) on jessie.. BUT re-adds tuning config from the past for mpm_prefork.conf (more SpareServers etc) that was not actually applied due to a bug [production]
00:10 <urandom> bootstrapping cassandra-c, restbase2018 -- T210843 [production]
2018-12-05 §
23:50 <ejegg> updated payments-wiki from 20595cca97 to b99cd0816e [production]
23:40 <ejegg> re-enabled fundraising queue consumer jobs [production]
23:33 <ejegg> updated fundraising CiviCRM from e757753a46 to 3e5d74f17e [production]
23:32 <ejegg> turned off fundraising queue jobs for base queue consumer logic update [production]
22:43 <jijiki> restarting pdfreder on scb* hosts in eqiad [production]
21:44 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@243a503]: Update mobileapps to 2f44362 (duration: 02m 47s) [production]
21:41 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@243a503]: Update mobileapps to 2f44362 [production]
21:41 <mdholloway> mobileapps deployment failed for group default03, rolling back and retrying [production]
21:39 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@243a503]: Update mobileapps to 2f44362 (duration: 18m 46s) [production]
21:39 <arlolra> Updated Parsoid to a6058e3 (T210647, T208360, T205333) [production]
21:37 <urandom> bootstrapping cassandra-b, restbase2018 -- T210843 [production]
21:23 <arlolra@deploy1001> Finished deploy [parsoid/deploy@5e9a496]: Updating Parsoid to a6058e3 (duration: 11m 36s) [production]
21:20 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@243a503]: Update mobileapps to 2f44362 [production]
21:12 <arlolra@deploy1001> Started deploy [parsoid/deploy@5e9a496]: Updating Parsoid to a6058e3 [production]
20:08 <banyek> repooling labsdb1010 - T210693 [production]
19:12 <cmjohnson1> cloudvirt1019 for an all inclusive part swap by HPE [production]
18:53 <sbisson@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: Fix formatting of help panel links (duration: 00m 47s) [production]
18:01 <jijiki> uploaded thumbor_6.3.2+git20170607-1+deb9u1 to stretch-wikimedia [production]
17:23 <hoo@deploy1001> Synchronized wmf-config/Wikibase.php: Enable Kartographer maps on testwikidatawiki (T184933) (duration: 00m 46s) [production]
17:22 <hoo@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: (no justification provided) (duration: 00m 46s) [production]
17:20 <hoo@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: (no justification provided) (duration: 00m 46s) [production]
17:19 <XioNoX> remove private IPs from codfw cloud-instance-transport1-b T207663 [production]
17:11 <XioNoX> add public IPs to codfw cloud-instance-transport1-b T207663 [production]
16:58 <XioNoX> re-deactivate ams-ix prefix list entry on cr2-esams [production]
16:53 <XioNoX> activate ams-ix prefix list entry on cr2-esams [production]
16:27 <jijiki> uploaded python-thumbor-wikimedia_2.2-1+deb9u1 to stretch-wikimedia [production]
16:18 <akosiaris@deploy1001> scap-helm zotero finished [production]
16:18 <akosiaris@deploy1001> scap-helm zotero cluster codfw completed [production]
16:18 <akosiaris@deploy1001> scap-helm zotero upgrade production -f zotero-values-codfw.yaml stable/zotero [namespace: zotero, clusters: codfw] [production]
16:17 <akosiaris@deploy1001> scap-helm zotero finished [production]
16:17 <akosiaris@deploy1001> scap-helm zotero cluster eqiad completed [production]
16:17 <akosiaris@deploy1001> scap-helm zotero upgrade production -f zotero-values-eqiad.yaml stable/zotero [namespace: zotero, clusters: eqiad] [production]
16:17 <akosiaris@deploy1001> scap-helm zotero finished [production]
16:17 <akosiaris@deploy1001> scap-helm zotero cluster eqiad completed [production]