|
2019-04-15
§
|
| 15:30 |
<onimisionipe> |
restarting prometheus-wmf-elasticsearch-exporter-9200 on all elastic nodes |
[production] |
| 15:28 |
<_joe_> |
systemctl reset-failed on ms-be1027, debmonitor session |
[production] |
| 15:27 |
<bd808> |
Updated to 26c7d0a "Fix URL to tool creation form" (T220947) |
[tools.admin] |
| 15:24 |
<Amir1> |
end of foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https (T219871) |
[production] |
| 15:01 |
<andrewbogott> |
moving tools-redis-1001 to eqiad1-r |
[tools] |
| 14:55 |
<gehel> |
deploying tilerator to maps1001 to validate deployment is working - T220982 |
[production] |
| 14:55 |
<Amir1> |
start of foreachwikiindblist wikidataclient extensions/Wikibase/lib/maintenance/populateSitesTable.php --force-protocol https (T219871) |
[production] |
| 14:43 |
<_joe_> |
running apply-config-tilerator on maps1001 |
[production] |
| 14:40 |
<_joe_> |
running apply-config-karthoterian on maps1001 |
[production] |
| 14:22 |
<cdanis> |
T220982 cdanis@cumin1001.eqiad.wmnet ~ % sudo cumin 'maps1*' 'sudo chmod -R a+r /srv/deployment/tilerator /srv/deployment/kartotherian' |
[production] |
| 14:21 |
<cdanis> |
cdanis@cumin1001.eqiad.wmnet ~ % sudo cumin 'maps1*' "disable-puppet 'bad permissions - T220982 - cdanis'" |
[production] |
| 14:18 |
<cdanis> |
cdanis@cumin1001.eqiad.wmnet ~ % sudo cumin 'maps*' 'sudo chmod -R a+r /srv/deployment/tilerator /srv/deployment/kartotherian' |
[production] |
| 14:18 |
<gehel> |
reseting permissions on maps server fir /srv/deployment/kartotherian and /srv/deplyoment/tilerator |
[production] |
| 14:04 |
<moritzm> |
rebooting ms-fe1005 for combined kernel/glibc/OpenSSL update |
[production] |
| 13:57 |
<jbond42> |
upgrading puppet 4 -> 5 and facter 2 -> 3 on mediawiki::canary_appserver, mediawiki::appserver::canary_api and cache::cache roles |
[production] |
| 13:56 |
<gehel> |
restart tilerator / kartotherian on all maps servers for openssl update |
[production] |
| 13:55 |
<godog> |
start ms-be1013 decom - T220590 |
[production] |
| 13:42 |
<godog> |
reboot ms-be1013 |
[production] |
| 13:09 |
<moritzm> |
installing wget security updates on trusty hosts |
[production] |
| 13:08 |
<arturo> |
T219626 add DB grants for keystone/nova/nova_api to clouddb2001-dev (codfw1dev) |
[admin] |
| 12:59 |
<moritzm> |
restarting archiva on archiva1001 for OpenJDK security update |
[production] |
| 12:50 |
<moritzm> |
restarting Apache on matomo1001 to pick up OpenSSL update |
[production] |
| 12:14 |
<moritzm> |
rolling restart of HHVM/Apache on deployment servers to pick up OpenSSL update |
[production] |
| 11:59 |
<fsero> |
pointing boron docker builds to the new registry temporarily (docker builds on boron might fail) |
[production] |
| 11:35 |
<Amir1> |
EU swat is done |
[production] |
| 11:26 |
<moritzm> |
rolling restart of HHVM/Apache on labweb* to pick up OpenSSL update |
[production] |
| 10:16 |
<Amir1> |
ores:8f01d40 going beta |
[releng] |
| 09:58 |
<moritzm> |
installing openssl1.0 security updates |
[production] |
| 09:18 |
<gehel> |
unbanning elastic1029 from cluster |
[production] |
| 08:58 |
<moritzm> |
updating mediawiki servers in eqiad to version 1.8.1 of the PHP extension for wikidiff |
[production] |
| 08:51 |
<hashar> |
castor: nuked /srv/jenkins-workspace/caches/castor-mw-ext-and-skins/master/mwselenium-quibble-docker # T220948 |
[releng] |
| 08:29 |
<onimisionipe> |
increase wal_keep_segments on codfw maps master |
[production] |
| 08:19 |
<moritzm> |
updating mediawiki servers in codfw to version 1.8.1 of the PHP extension for wikidiff |
[production] |
| 08:12 |
<Amir1> |
f63fddf going staging |
[ores] |
| 07:50 |
<Amir1> |
ladsgroup@mwmaint1002:~$ mwscript maintenance/initSiteStats.php --wiki=hywwiki --active (T220936) |
[production] |
| 05:31 |
<marostegui> |
Upgrade db1100 |
[production] |
| 05:07 |
<marostegui> |
powercycle mw1280 (crashed) |
[production] |
|
2019-04-14
§
|
| 16:23 |
<andrewbogott> |
moved all tools-worker nodes off of cloudvirt1015 and uncordoned them |
[tools] |
| 12:23 |
<lucaswerkmeister> |
kubectl delete pod quickcategories-654583560-xqip5 |
[tools.quickcategories] |
| 08:07 |
<Cam11598> |
1:06:50 AM <ChanServ> Flags +AV were set on rhinosf1 in #cvn-wp-en. |
[cvn] |
| 07:52 |
<Cam11598> |
12:51:25 AM <ChanServ> Flags +AV were set on BRPever in #cvn-wikidata. |
[cvn] |
| 07:51 |
<Cam11598> |
12:50:47 AM <ChanServ> Flags +AV were set on BRPever in #cvn-meta. |
[cvn] |
| 07:49 |
<Cam11598> |
12:49:04 AM <ChanServ> Flags +AV were set on Tulsi in #cvn-wikidata. |
[cvn] |
| 07:40 |
<Cam11598> |
12:40:10 AM <ChanServ> Flags +AV were set on Tulsi in #cvn-meta. |
[cvn] |
| 07:40 |
<Cam11598> |
12:38:56 AM <ChanServ> Flags +AV were set on Tulsi in #cvn-wp-en. |
[cvn] |
| 06:10 |
<ebernhardson> |
unban elastic1027 from eqiad-psi |
[production] |
| 05:36 |
<ebernhardson> |
unbanning elastic1027 after about half the shards left and load dropped |
[production] |
| 05:31 |
<ebernhardson> |
ban elastic1027 from elasticsearch-psi in eqiad |
[production] |
| 04:59 |
<ebernhardson> |
restart elasticsearch_6@production-searhc-psi-eqiad on elastic1027 due to 100% cpu for last 30+ minutes |
[production] |