|
2019-04-10
§
|
| 07:09 |
<jijiki> |
Rolling restart thumbor service |
[production] |
| 07:08 |
<jijiki> |
Upgrading Thumbor servers to python-thumbor-wikimedia to 2.4-1+deb9u1 |
[production] |
| 06:59 |
<marostegui> |
Deploy schema change on x1 master, with replication, lag will happen on x1 T217453 |
[production] |
| 06:59 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool x1 slaves T217453 (duration: 01m 13s) |
[production] |
| 05:52 |
<_joe_> |
setting both mwdebug200{1,2} to pooled = inactive to remove them from scap dsh list and allow deployments, T219989 |
[production] |
| 05:12 |
<_joe_> |
same on mwdebug2001 |
[production] |
| 05:11 |
<Cam11598> |
10:11:08 PM <ChanServ> Flags +AV were set on DannyS712 in #cvn-wp-en. |
[cvn] |
| 05:08 |
<_joe_> |
removing hhvm cache on mwdebug2002 |
[production] |
| 00:37 |
<Krinkle> |
last scap sync-file failed to mwdebug2002.codfw and mwdebug2001.codfw due to insufficient disk space |
[production] |
| 00:32 |
<andrewbogott> |
migrating tools-worker-1022, 1023, 1025, 1026 to eqiad1-r |
[tools] |
| 00:20 |
<krinkle@deploy1001> |
Synchronized php-1.33.0-wmf.25/resources/src/startup/: I3b9f1a13379a / Ie9db60e417cca (duration: 01m 01s) |
[production] |
|
2019-04-09
§
|
| 23:32 |
<bd808> |
Updated crontab to correct day of week (Wed==3) and hour (German summer time) |
[tools.taim-bot] |
| 23:26 |
<bd808> |
Test |
[tools.stashbot] |
| 23:14 |
<twentyafterfour@deploy1001> |
Pruned MediaWiki: 1.33.0-wmf.17 [keeping static files] (duration: 06m 03s) |
[production] |
| 23:08 |
<Krinkle> |
Reloading Zuul to deploy https://phabricator.wikimedia.org/T220561 |
[releng] |
| 22:31 |
<twentyafterfour@deploy1001> |
Finished scap: testwikis wikis to 1.33.0-wmf.25 refs T206679 (duration: 39m 59s) |
[production] |
| 22:19 |
<chaomodus> |
uploaded python-pynetbox to apt.wikimedia.org/stretch-wikimedia (T217072) |
[production] |
| 22:13 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@c0a2977]: Bring RB on restbase20(19|20) up to date - T208087 (duration: 02m 32s) |
[production] |
| 22:11 |
<mobrovac@deploy1001> |
Started deploy [restbase/deploy@c0a2977]: Bring RB on restbase20(19|20) up to date - T208087 |
[production] |
| 22:04 |
<bstorm_> |
added the new region on port 80 to the elasticsearch security group for stashbot |
[tools] |
| 20:43 |
<andrewbogott> |
moving tools-worker-1018, 1019, 1020, 1021 to eqiad1-r |
[tools] |
| 20:04 |
<andrewbogott> |
moving tools-k8s-etcd-03 to eqiad1-r |
[tools] |
| 19:54 |
<andrewbogott> |
moving tools-flannel-etcd-02 to eqiad1-r |
[tools] |
| 19:46 |
<herron> |
added myself to ldap group cn=archiva-deployers,ou=groups,dc=wikimedia,dc=org |
[production] |
| 19:36 |
<bstorm_> |
T220543 stopped webservice because it runs up load on any node it runs on in its current state and needs fixing before it is run |
[tools.montage-beta] |
| 19:28 |
<bstorm_> |
restarted webservice because it was hung in uninterruptible sleep and not working--causing very high load as well |
[tools.montage-beta] |
| 19:10 |
<twentyafterfour> |
branching 1.33.0-wmf.25 |
[production] |
| 18:53 |
<crusnov@deploy1001> |
Finished deploy [netbox/deploy@018d83e]: Minor fix to Netbox-Ganeti sync script (duration: 00m 52s) |
[production] |
| 18:52 |
<crusnov@deploy1001> |
Started deploy [netbox/deploy@018d83e]: Minor fix to Netbox-Ganeti sync script |
[production] |
| 18:50 |
<thcipriani> |
gerrit back |
[production] |
| 18:48 |
<thcipriani> |
gerrit restart |
[production] |
| 18:48 |
<thcipriani@deploy1001> |
Finished deploy [gerrit/gerrit@43d2d2e]: Gerrit update (cobalt) -- restart incoming (duration: 00m 10s) |
[production] |
| 18:47 |
<thcipriani@deploy1001> |
Started deploy [gerrit/gerrit@43d2d2e]: Gerrit update (cobalt) -- restart incoming |
[production] |
| 18:46 |
<thcipriani@deploy1001> |
Finished deploy [gerrit/gerrit@43d2d2e]: Gerrit update (gerrit2001 only) (duration: 00m 10s) |
[production] |
| 18:46 |
<thcipriani@deploy1001> |
Started deploy [gerrit/gerrit@43d2d2e]: Gerrit update (gerrit2001 only) |
[production] |
| 18:42 |
<volans> |
restart icinga on icinga1001 - T196336 |
[production] |
| 18:40 |
<ottomata> |
chowning files in analytics.wm.org/datasets/archive/public-datasets/ as stats:wikidev |
[analytics] |
| 18:38 |
<cdanis> |
T196336 cdanis@icinga1001$ sudo systemctl restart nsca |
[production] |
| 18:36 |
<andrewbogott> |
moving tools-worker-1016, tools-worker-1017 to eqiad1-r |
[tools] |
| 18:27 |
<crusnov@deploy1001> |
Finished deploy [netbox/deploy@4aa3e47]: Add node sync to Netbox-Ganeti sync script - T215229 (duration: 00m 57s) |
[production] |
| 18:26 |
<crusnov@deploy1001> |
Started deploy [netbox/deploy@4aa3e47]: Add node sync to Netbox-Ganeti sync script - T215229 |
[production] |
| 18:11 |
<gilles@deploy1001> |
Finished deploy [performance/asoranking@4c83130]: (no justification provided) (duration: 00m 03s) |
[production] |
| 18:11 |
<gilles@deploy1001> |
Started deploy [performance/asoranking@4c83130]: (no justification provided) |
[production] |
| 18:07 |
<urandom> |
bootstrapping cassandra-c, restbase2020 -- T208087 |
[production] |
| 18:05 |
<andrewbogott> |
migrating tools-k8s-etcd-02 to eqiad1-r |
[tools] |
| 18:00 |
<andrewbogott> |
migrating tools-flannel-etcd-01 to eqiad1-r |
[tools] |
| 17:59 |
<Krinkle> |
Reloading Zuul to deploy https://gerrit.wikimedia.org/r/502560 |
[releng] |
| 17:58 |
<gilles@deploy1001> |
Finished deploy [performance/asoranking@4c83130]: (no justification provided) (duration: 00m 02s) |
[production] |
| 17:58 |
<gilles@deploy1001> |
Started deploy [performance/asoranking@4c83130]: (no justification provided) |
[production] |
| 17:56 |
<elukey> |
restart keyholder-agent on deploy1001 to pick up new settings for analytics (+ arm all the keys) |
[production] |