|
2019-11-01
§
|
| 14:02 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:56 |
<moritzm> |
upgrading mwdebug2002 to PHP 7.2.24 for some smoke tests with the new build |
[production] |
| 12:18 |
<ema> |
pool cp5010 with ATS backend T227432 |
[production] |
| 11:59 |
<ema@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 11:56 |
<ema@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 11:21 |
<ema> |
depool cp5010 and reimage as text_ats T227432 |
[production] |
| 11:08 |
<effie> |
enable puppet mediawiki and prometheus servers |
[production] |
| 10:54 |
<effie> |
remove prometheus-hhvm-exporter package from mw* servers - T229792 |
[production] |
| 10:37 |
<moritzm> |
installing clamav security updates on mendelevium |
[production] |
| 10:33 |
<effie> |
Disable puppet on mediawiki and prometheus servers to remove hhvm exporters - T229792 |
[production] |
| 09:28 |
<moritzm> |
installing file security updates on jessie |
[production] |
| 09:21 |
<effie> |
depool mw1317 |
[production] |
| 09:19 |
<moritzm> |
installing golang-1.11 security updates |
[production] |
| 08:57 |
<moritzm> |
installing ruby-loofah security updates |
[production] |
| 08:17 |
<moritzm> |
installing libarchive security updates |
[production] |
| 01:58 |
<volker-e@deploy1001> |
Finished deploy [design/style-guide@4d8d085]: deploying design/style-guide with mobile layout improvements (duration: 00m 05s) |
[production] |
| 01:58 |
<volker-e@deploy1001> |
Started deploy [design/style-guide@4d8d085]: deploying design/style-guide with mobile layout improvements |
[production] |
| 01:21 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.4/resources/src/mediawiki.widgets/mw.widgets.UsersMultiselectWidget.js: T236460 mw.widgets.UsersMultiselectWidget: Fix property name (duration: 00m 54s) |
[production] |
|
2019-10-31
§
|
| 23:33 |
<Urbanecm> |
Evening SWAT done |
[production] |
| 23:26 |
<urbanecm@deploy1001> |
Synchronized php-1.35.0-wmf.4/extensions/CentralNotice/extension.json: SWAT: dcd3ec3: Fix error in CentralNoticeImpression schema (T236627) (duration: 00m 51s) |
[production] |
| 23:24 |
<urbanecm@deploy1001> |
Synchronized php-1.35.0-wmf.4/extensions/VisualEditor/: SWAT: 3686b82: Revert "Parse relative hrefs on image nodes like on regular links" (T237040) (duration: 00m 53s) |
[production] |
| 23:21 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 02bf4b8: Re-enable mobile editor A/B testing (T236337) (duration: 00m 52s) |
[production] |
| 23:13 |
<Urbanecm> |
Purge https://en.wikipedia.org/static/images/project-logos/bawiki* (T237035) |
[production] |
| 23:11 |
<urbanecm@deploy1001> |
Synchronized static/images/project-logos/: SWAT: 54ee973: Change bawiki logo to an anniversary one (T237035) (duration: 00m 53s) |
[production] |
| 23:04 |
<eileen> |
civicrm revision changed from d2045c6b98 to 1183915bde, config revision is 1a709a61aa |
[production] |
| 23:00 |
<mutante> |
replacing deployment keys for apache2secmod ; re-arming keyholder on deployment server |
[production] |
| 22:51 |
<XioNoX> |
Homer push to cr1/2-eqiad |
[production] |
| 22:17 |
<XioNoX> |
Homer push to cr1/2-codfw |
[production] |
| 22:14 |
<twentyafterfour@deploy1001> |
Finished deploy [design/style-guide@4d8d085]: testing deploy_design (duration: 00m 06s) |
[production] |
| 22:14 |
<twentyafterfour@deploy1001> |
Started deploy [design/style-guide@4d8d085]: testing deploy_design |
[production] |
| 22:12 |
<mutante> |
vega sudo find /srv/deployment/design/ -uid 498 -exec chown deploy-design:deploy-design {} \; |
[production] |
| 22:12 |
<twentyafterfour@deploy1001> |
deploy aborted: testing deploy_design (duration: 05m 07s) |
[production] |
| 22:12 |
<mutante> |
bromine sudo find /srv/deployment/design/ -uid 498 -exec chown deploy-design:deploy-design {} \; |
[production] |
| 22:07 |
<twentyafterfour@deploy1001> |
Started deploy [design/style-guide@4d8d085]: testing deploy_design |
[production] |
| 22:05 |
<twentyafterfour@deploy1001> |
Finished deploy [design/style-guide@4d8d085]: testing deploy_design (duration: 01m 30s) |
[production] |
| 22:04 |
<twentyafterfour@deploy1001> |
Started deploy [design/style-guide@4d8d085]: testing deploy_design |
[production] |
| 21:59 |
<mutante> |
deploy1001 - recreating deploy_design deployment key as ED25519 and with the correct comment (the comment matters and must match path to the file for keyholder) (T235677) |
[production] |
| 21:49 |
<mutante> |
deploy1001 keyholder restart, keyholder arm ... |
[production] |
| 21:46 |
<mutante> |
deploy1001 - move apach2modsec deployment key out of keyholder dir, keyholder arm to reload all other deployment keys including the new one for design (T235677) |
[production] |
| 21:32 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@9cac9ac]: Bump Parsoid-PHP traffic mirroring to 50% T235902 (duration: 13m 44s) |
[production] |
| 21:25 |
<robh> |
setting up ps1-b8-eqiad per T227543. it will reboot twice in the next 15 minutes, and then should start to clear up in icinga |
[production] |
| 21:18 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@9cac9ac]: Bump Parsoid-PHP traffic mirroring to 50% T235902 |
[production] |
| 20:35 |
<XioNoX> |
Homer push to all cr2-eqdfw - new NTP servers, remove border-in4 term unused-ips, add (unused) BGP_Wikimedia_pops, re-order ospf interfaces |
[production] |
| 20:27 |
<shdubsh> |
restarting logstash on logstash1008 to test level->severity filter selector |
[production] |
| 20:12 |
<XioNoX> |
Homer push to all msw* - new NTP servers - T237011 |
[production] |
| 20:07 |
<XioNoX> |
Homer push to all asw* - new NTP servers - T237011 |
[production] |
| 19:49 |
<XioNoX> |
Homer push to eqsin |
[production] |
| 19:49 |
<mutante> |
rsyncing home dirs from previous gerrit server cobalt to gerrit1001 |
[production] |
| 19:36 |
<fdans@deploy1001> |
Finished deploy [analytics/refinery@af91ce6]: deploying refinery, second attempt (duration: 06m 53s) |
[production] |
| 19:31 |
<XioNoX> |
Homer push to ulsfo |
[production] |