|
2018-09-21
§
|
| 23:47 |
<mutante> |
mwmaint1002 - rsyncing home dirs over from mwmaint1001 which synced them from terbium. still large old files from terbium here, like a NASA video :) (T201343) |
[production] |
| 23:25 |
<mutante> |
deleting now unused trusty instance relic (T204564) (/etc is backed up on new instance in /root/ just in case) |
[toolserver-legacy] |
| 23:17 |
<mutante> |
deleting unused instance trusty-update (T204558) |
[puppet] |
| 23:16 |
<mutante> |
deleting unused instance puppet-mailman (T204558) |
[puppet] |
| 23:12 |
<mutante> |
migration complete. shutting down trusty instance 'relic'. (T204564) |
[toolserver-legacy] |
| 22:40 |
<mutante> |
mwmaint1002 - scap pull |
[production] |
| 22:13 |
<mutante> |
mwmaint1002 - re-enabling puppet, now that it has mcrouter certs, that works (T201343) |
[production] |
| 22:13 |
<mutante> |
mwmaint1002 - re-enabling puppet, now that it has mcrouter certs, that works (201343) |
[production] |
| 22:12 |
<mutante> |
how to generate mcrouter certs, needed when you add any new mw server: https://wikitech.wikimedia.org/wiki/Mcrouter#Generate_certs_for_a_new_host |
[production] |
| 21:56 |
<mutante> |
sending all frozen messages from exim queue on old instance to clean up |
[toolserver-legacy] |
| 21:19 |
<mutante> |
update DNS zone entries and remove eqiad-r IP, revert to eqiad IP |
[toolserver-legacy] |
| 21:16 |
<mutante> |
associating floating IP (the existing one in eqiad) with relic-stretch-eqiad |
[toolserver-legacy] |
| 19:34 |
<andrewbogott> |
deleting project as per note on https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2018_Purge#OBSOLETE_discourse-wam |
[discourse-wam] |
| 19:29 |
<framawiki> |
deployed 4b01077 on quarry-web-01 T204964 |
[quarry] |
| 19:10 |
<XioNoX> |
shutdown cr3/4-ulsfo (not in prod) |
[production] |
| 18:19 |
<bawolff> |
deploy patch related to T197279 T204825 |
[production] |
| 18:08 |
<bblack> |
upgrade multatuli to gdnsd-2.99.1729-beta |
[production] |
| 18:06 |
<bblack> |
uploaded 2.99.1729-beta-1+wmf1 to stretch-wikimedia |
[production] |
| 18:06 |
<mutante> |
scp /etc/toolserver.aliases from relic to relic-stretch (exim legacy aliases for @toolserver.org mail address forwarding), not puppetized, contains personal email addresses |
[toolserver-legacy] |
| 17:59 |
<mutante> |
*.toolserver.org also moved from eqiad to eqiad-r region in cloud vps, which gave it new IP addresses |
[production] |
| 17:58 |
<mutante> |
toolserver.org and subdomains (wiki.toolserver, status.toolserver, stable.toolserver) legacy URLs have been switched to new stretch backend, away from trusty |
[production] |
| 17:50 |
<mutante> |
scp toolserver.org Letsencrypt key and cert from relic to relic-stretch instance for migration |
[toolserver-legacy] |
| 17:26 |
<marxarelli> |
adding jenkins node integration-slave-docker-1038 with 7 executors |
[releng] |
| 17:18 |
<bd808> |
Running `sudo maintain-meta_p --all-databases --purge` across labsdb10(09|10|11) for T201890 |
[admin] |
| 17:06 |
<mutante> |
relic-stretch: a2enmod ssl; starting apache |
[toolserver-legacy] |
| 16:58 |
<wikibugs> |
Updated channels.yaml to: 59937c1b4d3c31cd15c802831117b1a4c43c6b35 Add GCI 2018 work-board to #wikimedia-dev-africa |
[tools.wikibugs] |
| 16:47 |
<marxarelli> |
added new jenkins node integration-slave-docker-1037 with 7 executors |
[releng] |
| 16:37 |
<XioNoX> |
route ns2-v6 to multatuli.wikimedia.org on cr1/2-esams |
[production] |
| 16:33 |
<mutante> |
puppetmaster1001: cergen --base-path /srv/private/modules/secret/secrets/mcrouter/ --generate /etc/cergen/mcrouter.manifests.d |
[production] |
| 16:32 |
<mutante> |
puppetmaster1001: mcrouter_generate_certs --generate |
[production] |
| 16:29 |
<mutante> |
puppetmaster: running mcrouter_generate_certs to add an mcrouter cert for mwmaint1002 (T201343) https://wikitech.wikimedia.org/wiki/Mcrouter#Generate_certs_for_a_new_host |
[production] |
| 15:49 |
<marxarelli> |
replacing integration-slave-docker-1036 with new bigram instance |
[releng] |
| 15:48 |
<marxarelli> |
taking node integration-slave-docker-1035 offline due to unusually high steal cpu time and long build durations |
[releng] |
| 15:39 |
<bblack> |
upgrading gdnsd to 2.99.42 on authdns2001 |
[production] |
| 15:17 |
<marxarelli> |
integration-slave-docker-1035/1036 showing unusually high cpu steal and unusually long mean build durations |
[releng] |
| 15:16 |
<XioNoX> |
route ns2 to multatuli.wikimedia.org on cr1/2-eqiad |
[production] |
| 15:15 |
<marxarelli> |
taking integration-slave-docker-1036 offline due to unusually high cpu steal % trend |
[releng] |
| 15:13 |
<marxarelli> |
launching integration-slave-docker-1037 bigram instance |
[releng] |
| 15:00 |
<bstorm_> |
Installed backported packages to nfs server and demonstrated that it can be made to work by following a procedure to be documented in T203254 |
[wmcs-nfs] |
| 13:41 |
<Krenair> |
Created shinken-02 for T204562 |
[shinken] |
| 13:35 |
<ladsgroup@deploy1001> |
Finished deploy [ores/deploy@7b987a7]: Pass the real IP (T205087) (duration: 24m 10s) |
[production] |
| 13:11 |
<ladsgroup@deploy1001> |
Started deploy [ores/deploy@7b987a7]: Pass the real IP (T205087) |
[production] |
| 13:03 |
<Amir1> |
ores:7b987a7 is going beta |
[releng] |
| 13:01 |
<volans> |
wiping netbox DB to re-import it cleanly from racktables - T199083 |
[production] |
| 12:43 |
<banyek> |
adding wmf-pt-kill_2.2.20-1+wmf1 package for jessie |
[production] |
| 12:35 |
<arturo> |
cleanup stalled apt preference files (pinning) in tools-clushmaster-01 |
[tools] |
| 12:31 |
<banyek> |
adding wmf-pt-kill_2.2.20-1+wmf1 package for strech |
[production] |
| 12:24 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool pc1006 (duration: 00m 49s) |
[production] |
| 12:14 |
<arturo> |
T205078 same for {jessie,stretch}-wikimedia |
[tools] |