2701-2750 of 10000 results (86ms)
2019-04-25 §
06:14 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2080 after onsite maintenance to upgrade BIOS and firmware - T216240 (duration: 00m 54s) [production]
06:04 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Promote db2079 to s8 codfw master T220170 (duration: 00m 52s) [production]
05:47 <marostegui> Start changing topology to make db2079 s8 codfw master - T220170 [production]
05:28 <marostegui> Deploy schema change on db1103:3314 to fix revision table partitioning and indexing - T221782 [production]
05:27 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1103:3314 T221782 (duration: 00m 54s) [production]
01:43 <ejegg|afk> updated fundraising CiviCRM from 468f85e524 to 519fe8028e [production]
00:12 <maxsem@deploy1001> Synchronized php-1.34.0-wmf.1/extensions/CirrusSearch/includes/Maintenance/AnalysisConfigBuilder.php: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/CirrusSearch/+/506209/ (duration: 00m 54s) [production]
00:00 <eileen> process-control config revision is 0098b7a118 - adjust dedupe rule [production]
2019-04-24 §
22:46 <mutante> icinga-downtime -h ms-be2034 -r swift-rebalancing -d 86400 [production]
22:19 <mutante> deploying varnish/trafficserver change to cover www.wikiba.se (not prod yet) [production]
22:19 <mutante> icinga-downtime -h ms-be2039 -r swift-rebalancing -d 86400 [production]
21:31 <mutante> icinga-downtime -h ms-be2038 -r swift-rebalancing -d 86400 [production]
20:41 <mobrovac@deploy1001> Finished deploy [restbase/deploy@8a6b6fc]: Parsoid storage simplification step 1: switch Parsoid stashing to simple key/value - T215956 (duration: 20m 39s) [production]
20:21 <mobrovac@deploy1001> Started deploy [restbase/deploy@8a6b6fc]: Parsoid storage simplification step 1: switch Parsoid stashing to simple key/value - T215956 [production]
20:01 <mobrovac@deploy1001> Finished deploy [restbase/deploy@8a6b6fc] (dev-cluster): Switch Parsoid stashing to simple key/value (duration: 04m 18s) [production]
19:57 <mobrovac@deploy1001> Started deploy [restbase/deploy@8a6b6fc] (dev-cluster): Switch Parsoid stashing to simple key/value [production]
18:47 <mutante> pooled mw1297 as a new API server (T192457) [production]
18:46 <dzahn@cumin1001> conftool action : set/pooled=yes; selector: name=mw1297.eqiad.wmnet,cluster=api_appserver [production]
18:45 <mutante> mw1297 - scap pull [production]
18:17 <mutante> sudo icinga-downtime -h ms-be2031 -r swift-rebalancing -d 86400 [production]
17:52 <mutante> contint1001 - for logfile in $(find /var/log/zuul/ ! -name "*.gz"); do gzip $logfile; done to get more disk space (T207707) [production]
17:33 <mutante> contint1001 - apt-get clean for 1% more disk space [production]
17:23 <mutante> proton1001 - restarting proton service - low RAM caused facter/puppet fails (https://tickets.puppetlabs.com/browse/PUP-8048) freed memory and fixed puppet run (cc: T219456 T214975) [production]
16:33 <catrope@deploy1001> Synchronized php-1.34.0-wmf.1/extensions/GrowthExperiments/: Fix exceptions in Homepage logging (duration: 00m 56s) [production]
15:52 <herron> performing rolling restart of pybal on low-traffic eqiad/codfw lvs hosts [production]
15:32 <jijiki> Restarting php7.2-fpm on mw2* in codfw for 505383 and T211488 [production]
15:00 <herron> switching kibana lvs to source hash scheduler [production]
14:41 <jijiki> restart pdfrender on scb1002 [production]
14:28 <godog> being rollout rsyslog 8.1901.0-1 to jessie hosts - T219764 [production]
13:37 <marostegui> Poweroff db2080 for onsite maintenance - T216240 [production]
13:01 <jijiki> Restarting php7.2-fpm on mw13* for 505383 and T211488 [production]
12:36 <jijiki> restarting pdfrender on scb1004 [production]
12:23 <moritzm> rolling restart of Cassandra on restbase/eqiad to pick up Java security update [production]
11:59 <jijiki> Restarting php7.2-fpm on mw12* for 505383 and T211488 [production]
11:45 <gehel> restarting relforge for jvm ugprade [production]
11:33 <jbond42> security update ghostscript on scb jessie servers [production]
11:25 <jijiki> Restarting php7.2-fpm on mw-canary for 505383 and T211488 [production]
11:23 <ladsgroup@deploy1001> Finished deploy [ores/deploy@060fc37]: (no justification provided) (duration: 16m 18s) [production]
11:07 <ladsgroup@deploy1001> Started deploy [ores/deploy@060fc37]: (no justification provided) [production]
10:28 <akosiaris@deploy1001> scap-helm cxserver finished [production]
10:28 <akosiaris@deploy1001> scap-helm cxserver cluster staging completed [production]
10:28 <akosiaris@deploy1001> scap-helm cxserver upgrade -f cxserver-staging-values.yaml staging stable/cxserver [namespace: cxserver, clusters: staging] [production]
10:23 <jijiki> Restarting php-fpm on mw1238 for 505383 and T211488 [production]
09:58 <moritzm> installing rsync security updates on jessie [production]
08:44 <moritzm> rolling restart of Cassandra on restbase/codfw to pick up Java security update [production]
08:29 <godog> swift eqiad-prod: start decom for ms-be101[45] - T220590 [production]
08:17 <godog> bounce prometheus on bast5001 after migration and backfill [production]
08:04 <gehel@cumin1001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]
08:04 <gehel@cumin1001> START - Cookbook sre.elasticsearch.force-shard-allocation [production]
08:02 <gehel@cumin1001> END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) [production]