|
2019-05-31
§
|
| 10:54 |
<jynus> |
depool labsdb1010 for maintenance |
[production] |
| 10:47 |
<arturo> |
merging multiple commits to labs/private.git. We now require `puppet-merge --labsprivate` and people may not be yet aware of that |
[production] |
| 09:28 |
<jynus> |
stop and upgrade db2073 |
[production] |
| 09:11 |
<jynus> |
stop and upgrade db2095 (s2, s4, s6, s7) |
[production] |
| 08:33 |
<jynus> |
upgrade and restart db2065 |
[production] |
| 08:16 |
<jynus> |
depool labsdb1011 for maintenance |
[production] |
| 07:54 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1099 with low weight (duration: 00m 49s) |
[production] |
| 07:43 |
<_joe_> |
restarting php-fpm on canaries |
[production] |
| 07:24 |
<_joe_> |
repooling mw1348 |
[production] |
| 07:24 |
<jynus> |
upgrade and restart labsdb1009 |
[production] |
| 07:15 |
<_joe_> |
draining mw1348 from traffic |
[production] |
| 07:14 |
<jynus> |
depool labsdb1009 for maintenance |
[production] |
| 06:55 |
<jynus> |
upgrade and restart db2058 |
[production] |
| 06:33 |
<_joe_> |
repooled mw1348 |
[production] |
| 06:21 |
<jijiki> |
depool mw1348 |
[production] |
| 06:16 |
<_joe_> |
restarting php-fpm on mw1348 |
[production] |
| 00:08 |
<jgleeson> |
Updating civicrm from bb4acf3d8a to e028bfcd63 |
[production] |
|
2019-05-30
§
|
| 23:36 |
<XioNoX> |
remove BGP sessions to starhub on cr4-ulsfo (left the IXP) |
[production] |
| 23:19 |
<James_F> |
Pruned releng/quibble-fresnel:0.0.31-3 8ca03484a2c3 from contint1001 to save space |
[releng] |
| 23:02 |
<James_F> |
Updating docker-pkg files on contint1001 for +quibble-fresnel (0.0.31-4) |
[releng] |
| 22:59 |
<marxarelli> |
deleted 95 docker images from contint1001, freeing ~ 8G on / cc: T219850 |
[production] |
| 22:59 |
<XioNoX> |
add terms to drop specific icmp frag packets from cr1/2-eqiad - T224186 |
[production] |
| 22:53 |
<marxarelli> |
deleting stale docker images from contint1001, cc: T207707 T219850 |
[production] |
| 22:30 |
<wm-bot> |
<lucaswerkmeister> deployed ff3e7b2931 (add sort key hint to placeholder) |
[tools.quickcategories] |
| 22:25 |
<mutante> |
phab2001 / phab1003 - why is 'git status' in /srv/phab/phabricator unclean with lots of file deletions but also not identical |
[production] |
| 22:24 |
<mutante> |
phab2001 - scap pull - but it fails with directory /srv/mediawiki not found that's so wrong |
[production] |
| 22:20 |
<niharika29@deploy1001> |
Synchronized php-1.34.0-wmf.7/extensions/WikimediaEvents/: Avoid division by zero warnings T224686 (duration: 00m 49s) |
[production] |
| 22:19 |
<niharika29@deploy1001> |
Synchronized php-1.34.0-wmf.7/extensions/PageTriage/: Fix broken feed - T224693 (duration: 00m 51s) |
[production] |
| 22:17 |
<wm-bot> |
<lucaswerkmeister> deployed e3cd2871eb (sort key support and minor UI fix) |
[tools.quickcategories] |
| 22:12 |
<wm-bot> |
<lucaswerkmeister> git remote add github https://github.com/lucaswerkmeister/tool-quickcategories.git # work around T224677 |
[tools.quickcategories] |
| 21:33 |
<Krinkle> |
Jenkins admin says ""Your Jenkins data directory /var/lib/jenkins (AKA JENKINS_HOME) is almost full. You should act on it before it gets completely full."" |
[releng] |
| 21:27 |
<Krinkle> |
krinkle@mwmaint1002 Add 1 row to pagetriage_tags table on test2wiki db, based on PageTriageTagsPatch-recreated.sql. T224693, T189929 |
[production] |
| 21:27 |
<greg-g> |
back in business (ugh) |
[releng] |
| 21:21 |
<greg-g> |
doing the zuul deadlock dance again |
[releng] |
| 21:12 |
<Krinkle> |
krinkle@mwmaint1002 Add 1 row to pagetriage_tags table on testwiki db, based on PageTriageTagsPatch-recreated.sql. T224693, T189929 |
[production] |
| 21:11 |
<Krinkle> |
krinkle@mwmaint1002 Add 1 row to pagetriage_tags table on enwiki, based on PageTriageTagsPatch-recreated.sql. T224693, T189929 |
[production] |
| 21:10 |
<niharika29@deploy1001> |
Synchronized php-1.34.0-wmf.7/extensions/PageTriage: Bump wgPageTriageCacheVersion T224693 (duration: 00m 51s) |
[production] |
| 21:07 |
<XioNoX> |
add RPKI sessions on cr4-ulsfo - T220669 |
[production] |
| 20:51 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/513168 |
[releng] |
| 20:39 |
<twentyafterfour> |
phabricator: restart ssh-phab.service |
[production] |
| 19:49 |
<mutante> |
sodium (mirrors) - sudo -u mirror /usr/local/sbin/update-ubuntu-mirror |
[production] |
| 18:49 |
<Urbanecm> |
Morning SWAT finished |
[production] |
| 18:47 |
<urbanecm@deploy1001> |
Synchronized php-1.34.0-wmf.7/extensions/GrowthExperiments/: [[:gerrit:513300|QuestionPoster: Correctly set timestamp when question is posted]] (T223338) (duration: 00m 51s) |
[production] |
| 18:26 |
<mutante> |
phab1003 - switch 'vcs' user to 'NP' to match phab1001 setup and then /srv/phab/phabricator# ./bin/config set diffusion.ssh-user vcs (T224677) |
[production] |
| 18:24 |
<XioNoX> |
bounce eqord-ulsfo interface to try to fix BFD sessions |
[production] |
| 18:12 |
<Krinkle> |
Running `php7adm /opcache-free` on mw1348 and mw1321, T224491 |
[production] |
| 18:12 |
<Krinkle> |
Running `php7adm /opcache-free` on mw1348 and mw1321 |
[production] |
| 18:11 |
<Krinkle> |
mw1348 (recent api/php72 100% experiment) shows signs of corruption |
[production] |
| 18:11 |
<Krinkle> |
mw1321 php7.2 shows signs of corruption for over 2 hours – https://phabricator.wikimedia.org/T224491#5224464 |
[production] |
| 18:03 |
<krinkle@deploy1001> |
Synchronized wmf-config/arclamp.php: (no justification provided) (duration: 00m 53s) |
[production] |