|
2019-07-10
ยง
|
| 22:59 |
<jforrester@deploy1001> |
scap failed: average error rate on 3/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/db09a36be5ed3e81155041f7d46ad040 for details) |
[production] |
| 22:57 |
<jforrester@deploy1001> |
Synchronized php-1.34.0-wmf.13/includes/user/User.php: T227688 User: support setting custom fields + array autocreation in non-existent field (duration: 00m 58s) |
[production] |
| 22:46 |
<shdubsh> |
downgrading cp4031 to mtail_3.0.0~rc5-1~bpo9+1wmf1 to fix varnishmtail T225604 |
[production] |
| 22:46 |
<jforrester@deploy1001> |
Synchronized w: T156319 Remove /w/skin-1.5 symlink (duration: 00m 58s) |
[production] |
| 22:16 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T212865 Stop configuring ZeroBanner and ZeroPortal, unused (duration: 00m 58s) |
[production] |
| 22:06 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T212865 Drop the ability to use ZeroBanner and ZeroPortal from production (duration: 00m 57s) |
[production] |
| 22:03 |
<jforrester@deploy1001> |
Synchronized wmf-config/mobile.php: T212865 Drop the ability to use ZeroBanner and ZeroPortal from production, mobile code (duration: 00m 57s) |
[production] |
| 21:59 |
<jforrester@deploy1001> |
Synchronized w/robots.php: T212865 Drop the special treatment for Wikipedia Zero (duration: 00m 58s) |
[production] |
| 21:55 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T212865 Drop the Wikipedia Zero debug log channel (duration: 00m 58s) |
[production] |
| 21:51 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T187716 Drop all zerowiki configuration (duration: 00m 58s) |
[production] |
| 21:50 |
<mutante> |
mwdebug1002 - php7adm /opcache-free because icinga showed a warning for opcache free space below 100MB |
[production] |
| 21:49 |
<jforrester@deploy1001> |
Synchronized dblists/: T187716 Mark zerowiki as deleted in dblists (duration: 01m 00s) |
[production] |
| 21:41 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T212865 Disable ZeroBanner on all wikis (duration: 00m 59s) |
[production] |
| 21:36 |
<mutante> |
mw1235 - restarting hhvm (socket timeout alert in icinga since about 1.5h) |
[production] |
| 21:35 |
<mutante> |
mw1290 - restarting hhvm (socket timeout alert in icinga since about 5h) |
[production] |
| 19:45 |
<hoo> |
Updated the Wikidata property suggester with data from the 2019-07-01 JSON dump and applied the T132839 workarounds |
[production] |
| 19:32 |
<otto@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Produce recentchange stream to eventgate-main - T211248 (duration: 00m 57s) |
[production] |
| 19:26 |
<otto@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Use wgEventServiceStreamConfig to configure wgRCFeeds eventbus. No-op in prod. - T211248 (duration: 00m 58s) |
[production] |
| 19:05 |
<jiji@deploy1001> |
Finished deploy [cpjobqueue/deploy@8761480]: Migrating rest of hightraffic jobs to PHP7 - T219150 (duration: 01m 00s) |
[production] |
| 19:04 |
<jiji@deploy1001> |
Started deploy [cpjobqueue/deploy@8761480]: Migrating rest of hightraffic jobs to PHP7 - T219150 |
[production] |
| 18:15 |
<jforrester@deploy1001> |
Synchronized php-1.34.0-wmf.13/includes/Linker.php: T227656 Fix visibility of IPs that aren't suppressed (duration: 00m 59s) |
[production] |
| 17:54 |
<twentyafterfour> |
phabricator: hotfixing fatal error by pulling upstream fix ( see https://secure.phabricator.com/D20644 ) |
[production] |
| 16:09 |
<Urbanecm> |
Morning SWAT done |
[production] |
| 16:09 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:521865|Change bawikibooks logo to correct one according to community wish]] (2/2, T227418) (duration: 00m 58s) |
[production] |
| 16:07 |
<Urbanecm> |
Purged two urls for T227418 |
[production] |
| 16:06 |
<urbanecm@deploy1001> |
Synchronized static/images/project-logos/: SWAT: [[:gerrit:521865|Change bawikibooks logo to correct one according to community]] (1/2, T227418) (duration: 01m 16s) |
[production] |
| 16:04 |
<urbanecm@deploy1001> |
Synchronized dblists/commonsuploads.dblist: SWAT: [[:gerrit:521497|Disable local uploads on wuuwiki]] (T226764) (duration: 00m 58s) |
[production] |
| 15:23 |
<ema> |
cp-ulsfo: upgrade varnish to 5.1.3-1wm11 T227672 |
[production] |
| 15:08 |
<ema> |
restart wb2-phab wikibugs job |
[production] |
| 14:51 |
<ema> |
upload varnish 5.1.3-1wm11 to stretch-wikimedia T227672 |
[production] |
| 14:42 |
<godog> |
reimage ms-be2022 - T227667 |
[production] |
| 14:03 |
<jbond42> |
copy puppetdb-termini 4.4.0-1~wmf2 from stretch-wikimedia to jessie-wikimedia |
[production] |
| 13:47 |
<ema> |
cp hosts: cleanup WP zero leftovers T213769 |
[production] |
| 13:22 |
<godog> |
reset ilo on ms-be2022 - bios can't talk to it on boot |
[production] |
| 12:49 |
<godog> |
reboot ms-be2022 - T225713 |
[production] |
| 11:53 |
<Urbanecm> |
Purged 14 urls for T211413 |
[production] |
| 11:51 |
<Urbanecm> |
Purged 24 urls for T227635 |
[production] |
| 11:11 |
<Urbanecm> |
EU SWAT done |
[production] |
| 11:11 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Remove autopromote to patroller on testwiki (T168718) (duration: 00m 58s) |
[production] |
| 11:10 |
<urbanecm@deploy1001> |
Synchronized static/images/project-logos/: SWAT: Several logo changes (T227635 T211413) (duration: 01m 00s) |
[production] |
| 11:09 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: Remove fawikiquote HD logo (T211413) (duration: 00m 57s) |
[production] |
| 11:07 |
<urbanecm@deploy1001> |
sync-file aborted: SWAT: Several logo changes (T227635 T211413) (duration: 00m 20s) |
[production] |
| 11:06 |
<urbanecm@deploy1001> |
Synchronized docroot/noc/conf/highlight.php: SWAT: [[:gerrit:521576|Fix non-working "raw text" links on noc.wikimedia.org web pages]] (T227606) (duration: 01m 02s) |
[production] |
| 09:57 |
<moritzm> |
re-enabled puppet on hosts using acme_chief::cert for reboots of acmechief hosts (actually did that 20 minutes ago, but missed to log earlier) |
[production] |
| 09:54 |
<jynus> |
disabling puppet on prometheus* hosts for upcoming deploy |
[production] |
| 09:38 |
<fsero> |
doing the same on ms-be1030 |
[production] |
| 09:37 |
<fsero> |
docker-registry: running manual only once swift-container-sync on ms-be2019 |
[production] |
| 09:36 |
<moritzm> |
rearmed keyholder on acmechief1001 |
[production] |
| 09:29 |
<moritzm> |
rebooting acmechief1001 to pick up MDS-enabled qemu |
[production] |
| 09:25 |
<moritzm> |
rearmed keyholder on acmechief2001 |
[production] |