2019-05-09
§
|
11:18 |
<jbond42> |
starting puppetdb on puppetdb2001 |
[production] |
11:15 |
<jbond42> |
run sudo apt-get install puppetdb on puppetdb2001 |
[production] |
11:14 |
<jbond42> |
ran the folloowing on puppetdb2001 sudo apt-get install facter=2.4.6-1 puppet=4.8.2-5 |
[production] |
11:14 |
<jbond42> |
ran the folloowing on puppetmaster200{1,2} sudo apt-get install facter=2.4.6-1 puppet=4.8.2-5 puppet-master puppet-master-passenger |
[production] |
11:04 |
<_joe_> |
disabling puppet across the fleet |
[production] |
11:02 |
<volans> |
stopped ircecho to avoid spam |
[production] |
10:43 |
<marostegui> |
Stop MySQL on db1081 |
[production] |
10:41 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1081 (duration: 00m 57s) |
[production] |
10:29 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Give API traffic to db1129 (new host on s2) (duration: 00m 57s) |
[production] |
10:15 |
<_joe_> |
restarting low-traffic pybals in codfw, eqiad |
[production] |
10:05 |
<akosiaris> |
restart proton on proton1001. Host Out of memory T214975 |
[production] |
09:57 |
<ariel@deploy1001> |
Finished deploy [dumps/dumps@ab56fdd]: reduce sleep time between dumps of adds-changes wikis still more (retry) (duration: 00m 06s) |
[production] |
09:57 |
<ariel@deploy1001> |
Started deploy [dumps/dumps@ab56fdd]: reduce sleep time between dumps of adds-changes wikis still more (retry) |
[production] |
09:54 |
<ariel@deploy1001> |
Finished deploy [dumps/dumps@ab56fdd]: reduce sleep time between dumps of adds-changes wikis still more (duration: 00m 06s) |
[production] |
09:54 |
<ariel@deploy1001> |
Started deploy [dumps/dumps@ab56fdd]: reduce sleep time between dumps of adds-changes wikis still more |
[production] |
09:31 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1129 (new host on s2) (duration: 00m 57s) |
[production] |
09:29 |
<fsero@puppetmaster1001> |
conftool action : set/pooled=true; selector: dnsdisc=docker-registry,name=codfw |
[production] |
09:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1076 and db1129 (new host on s2) (duration: 00m 56s) |
[production] |
09:12 |
<godog> |
bounce rsyslog on lithium |
[production] |
09:01 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1076 and db1129 (new host on s2) (duration: 00m 56s) |
[production] |
08:49 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1076 and db1129 (new host on s2) (duration: 00m 56s) |
[production] |
08:38 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1076 and db1129 (new host on s2) (duration: 00m 57s) |
[production] |
08:28 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: More traffic to db1076 (duration: 00m 55s) |
[production] |
08:23 |
<elukey> |
upload uwsgi 2.0.14+20161117-3+deb9u2+wmf1 packages to stretch-wikimedia - T212697 |
[production] |
08:21 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Pool db1129 with low weight on s2 - T222682 (duration: 00m 56s) |
[production] |
08:13 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1076 (duration: 00m 56s) |
[production] |
08:07 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Provision db1129, db2104, db2107, db2108 T222772 T222682 (duration: 00m 57s) |
[production] |
08:06 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Provision db1129, db2104, db2107, db2108 T222772 T222682 (duration: 00m 59s) |
[production] |
07:54 |
<moritzm> |
installing jquery security updates for stretch |
[production] |
07:50 |
<elukey> |
roll restart HDFS masters on an-master100[1,2] to pick up new logging settings |
[production] |
07:23 |
<moritzm> |
installing twitter-bootstrap3 security updates |
[production] |
06:53 |
<_joe_> |
restarted nagios-nrpe-server on proton1001 |
[production] |
05:58 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Clarify disk status for db2103, db2112, db2116 (duration: 00m 58s) |
[production] |
05:29 |
<marostegui> |
Stop replication on db2098:s2 |
[production] |
05:25 |
<marostegui> |
Stop MySQL on db1076 |
[production] |
05:22 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1076 (duration: 00m 57s) |
[production] |
05:09 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Pool db2103, db2112 and db2116 into s1 T222772 (duration: 01m 41s) |
[production] |
05:07 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Pool db2103, db2112 and db2116 into s1 T222772 (duration: 01m 22s) |
[production] |
04:55 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool pc1007 (duration: 00m 59s) |
[production] |
00:57 |
<twentyafterfour> |
stopped phd, now running `puppet agent --test` manually on phab1001 |
[production] |
00:08 |
<twentyafterfour> |
phabricator upgrade successful |
[production] |
00:04 |
<twentyafterfour> |
starting phabricator deployment, momentary downtime expected (~1 minute) |
[production] |