tag:blogger.com,1999:blog-39914249650722006002024-03-05T03:17:50.293-07:00Brian's Virtually Useful BlogSRE, VMware Virtualization, vSphere, vCD, ESX, Configuration Management, Microsoft AD, Security, Networking, and about anything else..Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.comBlogger220125tag:blogger.com,1999:blog-3991424965072200600.post-5835379115549562572023-04-19T15:08:00.003-06:002023-04-19T15:13:30.418-06:00Why Bitcoin Needs to Go to Zero: A Comprehensive Look at its Negative Impact<p><b><i> I wanted to play with ChatGPT and have it create a blog post. Here is my its first creation with some input/guidance from myself.</i></b></p><br /><br />Bitcoin has been touted as a revolutionary technology that can transform the way we store and transfer value. However, there are numerous reasons why it needs to go to zero. While the underlying technology behind Bitcoin, known as blockchain, is genuinely innovative and potentially transformative, the issue with Bitcoin is that it is a bad use case for this technology. In this blog post, we will examine the various reasons why Bitcoin needs to go to zero and why blockchain technology should be used in more responsible and sustainable ways.<br /><br />1. Environmental Damage<br /><br />The process of mining Bitcoin is incredibly energy-intensive, consuming vast amounts of electricity and contributing to climate change. This high energy consumption is a significant problem, with studies showing that Bitcoin's energy usage is equivalent to that of a small country. The environmental damage caused by Bitcoin mining is already apparent and is set to become more severe unless we take action. Bitcoin mining in China alone could produce enough greenhouse gas emissions to raise the country's temperature by 2 degrees Celsius by 2033.<br /><br />2. Facilitation of Illegal Activities<br /><br />Bitcoin is often associated with illegal activities such as money laundering, drug trafficking, and terrorism financing. Its anonymity and lack of regulation make it an attractive option for criminals looking to evade detection and launder money. The prevalence of illegal activities associated with Bitcoin means that it poses a significant threat to global security.<br /><br />3. High Volatility<br /><br />Bitcoin's high volatility makes it a risky investment option, with prices fluctuating wildly over short periods. This volatility not only makes it difficult to predict the value of Bitcoin but also increases the risk of financial losses. Bitcoin's volatility can lead to financial instability, making it a significant threat to the global economy.<br /><br />4. Lack of Regulation<br /><br />Bitcoin operates outside of traditional financial systems, making it difficult to regulate effectively. This lack of regulation has led to numerous frauds and scams in the Bitcoin market, resulting in significant financial losses for individuals. Bitcoin's lack of regulation also makes it challenging to prevent illegal activities associated with it.<br /><br />5. Harming Societal Well-being<br /><br />Bitcoin's high energy consumption, illegal activities, financial instability, and lack of regulation are all factors that can harm societal well-being. The damage caused by Bitcoin extends beyond financial losses and environmental degradation, creating a world where criminal activities are facilitated, and financial instability is rampant.<br /><br />It is important to note that the underlying technology behind Bitcoin, blockchain, is a genuinely innovative and potentially transformative technology. Blockchain has the potential to revolutionize a range of industries, from finance to healthcare, by providing a secure, transparent, and decentralized way to store and transfer data.<br /><br />However, the issue with Bitcoin is that it is a bad use case for this technology. While the idea of a decentralized currency that operates outside of traditional financial systems is certainly compelling, the reality is that Bitcoin has become a vehicle for speculation and illegal activities, rather than a useful currency or payment system. This has resulted in the negative consequences mentioned earlier, such as environmental damage, financial instability, and harm to societal well-being.<br /><br />In conclusion, while blockchain technology has tremendous potential, Bitcoin is a bad implementation of it. It is time for us to recognize the shortcomings of Bitcoin and focus our efforts on finding more responsible and sustainable applications of blockchain technology. By doing so, we can ensure that the benefits of blockchain are realized while mitigating the negative consequences associated with Bitcoin. It is time for Bitcoin to go to zero, and for us to move on to more viable and sustainable uses of blockchain technology.<br /><div class="group w-full text-gray-800 dark:text-gray-100 border-b border-black/10 dark:border-gray-900/50 bg-gray-50 dark:bg-[#444654]" style="--tw-bg-opacity: 1; --tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(59,130,246,0.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-text-opacity: 1; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgba(0, 0, 0, 0.1); border-image: initial; border-style: solid; border-width: 0px 0px 1px; box-sizing: border-box; color: rgba(52,53,65,var(--tw-text-opacity)); width: 1108px;"><div class="text-base gap-4 md:gap-6 md:max-w-2xl lg:max-w-xl xl:max-w-3xl p-4 md:py-6 flex lg:px-0 m-auto" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(59,130,246,0.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; display: flex; gap: 1.5rem; line-height: 1.5rem; margin: auto; max-width: 48rem; padding: 1.5rem 0px;"><div class="relative flex w-[calc(100%-50px)] flex-col gap-1 md:gap-3 lg:w-[calc(100%-115px)]" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(59,130,246,0.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; display: flex; flex-direction: column; gap: 0.75rem; position: relative; width: calc(100% - 115px);"><div class="flex justify-between lg:block" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(59,130,246,0.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; justify-content: space-between;"><div class="text-gray-400 flex self-end lg:self-center justify-center mt-2 gap-2 md:gap-3 lg:gap-1 lg:absolute lg:top-0 lg:translate-x-full lg:right-0 lg:mt-0 lg:pl-2 visible" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(59,130,246,0.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-text-opacity: 1; --tw-translate-x: 100%; --tw-translate-y: 0; align-self: center; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; color: rgba(172,172,190,var(--tw-text-opacity)); display: flex; gap: 0.25rem; justify-content: center; margin-top: 0px; padding-left: 0.5rem; position: absolute; right: 0px; top: 0px; transform: translate(var(--tw-translate-x),var(--tw-translate-y)) rotate(var(--tw-rotate)) skewX(var(--tw-skew-x)) skewY(var(--tw-skew-y)) scaleX(var(--tw-scale-x)) scaleY(var(--tw-scale-y)); visibility: visible;"><button class="p-1 rounded-md hover:bg-gray-100 hover:text-gray-700 dark:text-gray-400 dark:hover:bg-gray-700 dark:hover:text-gray-200 disabled:dark:hover:text-gray-400" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(59,130,246,0.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; appearance: button; background-image: none; border-color: rgb(217, 217, 227); border-radius: 0.375rem; border-style: solid; border-width: 0px; cursor: pointer; line-height: inherit; margin: 0px; padding: 0.25rem;"><svg class="h-4 w-4" fill="none" height="1em" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" stroke="currentColor" viewbox="0 0 24 24" width="1em" xmlns="http://www.w3.org/2000/svg"><path d="M10 15v4a3 3 0 0 0 3 3l4-9V2H5.72a2 2 0 0 0-2 1.7l-1.38 9a2 2 0 0 0 2 2.3zm7-13h2.67A2.31 2.31 0 0 1 22 4v7a2.31 2.31 0 0 1-2.33 2H17"></path></svg></button></div></div></div></div></div>Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-2097510647762750682022-08-19T17:09:00.015-06:002022-08-19T17:26:03.340-06:00Virtual Transparent Firewall not working<p>I have been successfully using a <a href="https://www.pfsense.org/download/" target="_blank">pfSense</a> community edition firewall to protect my home test lab. My local ISP delivers me a subnet of addresses directly so I need to leverage a transparent firewall or "bridge" to protect the lab. After rebuilding a few pieces of the lab I restored my pfSense configuration to a new host/VM and found that no traffic was passing. I did a packet capture and did not see any communication traffic. I thought that *something* must be blocking the traffic before it gets to the pfSense transparent firewall VM. A lightbulb went off in my head back to my VMware architecture days about the 3 security settings you can set on a virtual network switch, promiscuous mode being the easiest to remember. I played with turning these on/off 1 by 1 and found I needed both Promiscuous mode and Forged transmits security turned off (Setting to Accept) for this pfSense transparent firewall VM to operate correctly. Obviously turning these features off does open your ESX Host up to accepting more (possibly malicious) packets, but the ESX host is simply passing the packets along to the VM(s) attached to that Network on the host. You can limit your exposure by only having a single VM on that host's "raw internet" network and a single (same) VM attached to the inside "filtered internet" network. Assuming you trust pfSense to do its job, turning off these features should work for most home use cases.</p><p><br /></p><p></p><div class="separator" style="clear: both; text-align: center;"><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/a/AVvXsEg31xyza5-IAhd8CH0HMgQ1Qf9yndJLS-XtV7n1jpkxRESRnXjDgKadzWzMUWfQyaZoX946e1Xgywfm87mGpe3cOKVhE43HiIo8GoiSkVZoJtfEjhN3mtORvJAu6AWst-uZTOjxgWuCX88eCsQ8jDm2oxPPWRr3P_72EHV4s5HDtvAy0H9-P2EBCxVzxg" style="margin-left: 1em; margin-right: 1em;"><img alt="" data-original-height="208" data-original-width="974" height="103" src="https://blogger.googleusercontent.com/img/a/AVvXsEg31xyza5-IAhd8CH0HMgQ1Qf9yndJLS-XtV7n1jpkxRESRnXjDgKadzWzMUWfQyaZoX946e1Xgywfm87mGpe3cOKVhE43HiIo8GoiSkVZoJtfEjhN3mtORvJAu6AWst-uZTOjxgWuCX88eCsQ8jDm2oxPPWRr3P_72EHV4s5HDtvAy0H9-P2EBCxVzxg=w484-h103" width="484" /></a></div><br /><br /></div><br /><br /><p></p>Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-34469155518454886612022-02-03T16:50:00.002-07:002022-02-03T18:41:17.103-07:00yum update failing on Google Cloud VM<p> I use google cloud's smallest VM for hosting my own DNS servers. I use the f1-micro instances that are very limited in memory and cpu, but cheap!</p><p>During a regular yum update I received the following error and my instance (VM) failed to reboot.</p><p>During the Cleanup part of yum update the google-cloud-sdk gave me this error:</p><p>/var/tmp/rpm-tpm.rdz2f9: line 4: 11963 Killed gcloud components post-process --force-recomplile warning: %postrun(google-cloud-sdk-360.0.0-1.x86_64) scriptlet failed, exit status 137 Non-fatal POSTUN scriptlet failure in rpm package google-cloud-sdk-360.0.0-1.x86_64</p><p>I read on this post - <a href="https://stackoverflow.com/questions/40163733/upgrading-google-cloud-sdk-fails-on-configure">https://stackoverflow.com/questions/40163733/upgrading-google-cloud-sdk-fails-on-configure</a> that this person had the same issue to due using the smallest GCP instance size, but they chose to stop some processes to free up memory before they did an update.</p><p>My resolution was to first build a new VM with a previous snapshot of the VM before it was messed up by the update.</p><p>Just for good measure I did some yum cleanup with yum-utils before anything else, then update the single google-cloud-sdk, before updating the rest without error.</p><p>sudo yum install yum-utils</p><p>sudo package-cleanup --dupes --noplugins</p><p>sudo yum clean all</p><p>sudo yum clean dbcache (probably redundant)</p><p>sudo yum update google-cloud-sdk (this took a long time)</p><p>sudo yum update (update all the other pieces)</p><p>After this everything was happy!</p><p><br /></p><p><br /></p>Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-70067526008167523152021-05-05T20:00:00.012-06:002021-05-05T21:56:43.045-06:00 How to setup DNS forwarding for specific domains (i.e. home.lab)<p></p><div class="separator" style="clear: both; text-align: center;"><br /></div><p></p><p class="p2" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px; min-height: 16px;">I recently purchased a USG-3P (Unified Secure Gateway) Ubiquiti Unify Home router.</p><p class="p1" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px;">With its DHCP, everything points to this device as the DNS server for the house. I like this because I don't want to depend on my home lab being up for DNS (Internet) to be working for basic name resolution. However, I want my xyz.home.lab domains to resolve. I did a bunch of googling and found I needed to modify the <span class="s1" style="font-family: "Helvetica Neue"; font-size: 11.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal;"><i>config.gateway.json </i></span>file, but I couldn't find it on the appliance.<span class="Apple-converted-space"> </span>I then found this <a href="https://help.ui.com/hc/en-us/articles/115010913367-EdgeRouter-DNS-Forwarding-Setup-and-Options"><span class="s2" style="color: #0563c1;">article</span></a>, so I just needed to SSH into my USG and run a command such as ---<span class="Apple-converted-space"> </span></p><p class="p1" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px;">set service dns forwarding options server=/lab.dns/172.16.1.10</p><p class="p1" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px;">This seemed great, but the command gave me “invalid command”, which it turns out you need to go into configure mode first on the USG,<span class="Apple-converted-space"> </span>just type ‘configure’ and hit enter and then run the command above.<span class="Apple-converted-space"> </span>After that you type commit, then save.<span class="Apple-converted-space"> </span></p><p class="p1" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px;">This almost makes my home work, but my primary mac machine is always VPN’d into work, and all DNS requests are sent there, so my home.lab still doesn’t work from this machine.</p><p class="p1" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px;">On my mac I needed to mkdir /etc/resolver, then in the /etc/resolver directory, then create a file named home.lab</p><p class="p1" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px;">Adding the following lines ----</p><p class="p1" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px;">nameserver 172.16.1.10</p><p class="p1" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px;">nameserver 172.16.1.11</p><p class="p1" style="font-size: 13.5px; font-stretch: normal; font-variant-east-asian: normal; font-variant-numeric: normal; line-height: normal; margin: 5px 0px;">After I saved that, I can now resolve my home.lab dns from everywhere, and I get the bonus of being able to still work just fine if the lab (DNS) is down.</p>Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-77563030194545549082021-04-29T16:32:00.004-06:002021-04-29T16:35:15.048-06:00Team Rules<p> I dug this up from many years ago, just as good today as it was then!</p><p><br /></p><p class="title" style="font-family: -apple-system, system-ui, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif; font-size: 14px; font-weight: bold; margin: 0px; padding: 0px;"><span style="color: white;">Team Rules - by Brian Smith</span></p><p><span class="aui-icon aui-icon-small aui-iconfont-warning confluence-information-macro-icon" face="-apple-system, system-ui, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif" style="background-position: 0px 0px; background-repeat: no-repeat; border: none; display: block; font-size: 14px; height: 16px; left: 10px; line-height: 20px; margin: 0px; padding: 0px; position: absolute; text-indent: -999em; top: 12px; vertical-align: text-bottom; width: 16px;"></span></p><div class="confluence-information-macro-body" style="font-family: -apple-system, system-ui, "Segoe UI", Roboto, Oxygen, Ubuntu, "Fira Sans", "Droid Sans", "Helvetica Neue", sans-serif; font-size: 14px; margin: 0px; padding: 0px;"><ol style="margin: 0px;"><li><span style="color: white;"><em><u>It's all about the customers!</u></em> We must provide a reliable and cost effective solution!</span></li><li><span style="color: white;">If there are customers on it, then it is in production and will be treated as production until the customers are off and it is NOT in production.</span></li><li><span style="color: white;">You can't say that's not my job, you may direct someone to the appropriate person or group if you can’t help them.</span></li><li><span style="color: white;">All Negative customer experiences must be escalated, no matter who you believe is responsible</span></li><li><span style="color: white;">If a project deadline is in jeopardy, you must escalate immediately.</span></li><li><span style="color: white;">There is no excuse use official methods (i.e. Open an Incident in the official ticketing tool)</span></li><li><span style="color: white;">When you are on call, you must be available to work on an issue.</span></li><li><span style="color: white;">Document before you execute and open a change for all changes (use official tools)</span></li><li><span style="color: white;">Do your best to not directly delete anything, take offline for 3-7 days when possible</span></li></ol></div>Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-74505904558680499682021-03-24T12:26:00.004-06:002021-03-24T12:26:53.617-06:00SRE BUZZWORD BINGO<p> ok, it's not bingo, but it is a crossword</p><p><a href="https://crosswordlabs.com/view/sre-buzzword" target="_blank">https://crosswordlabs.com/view/sre-buzzword </a></p><p>Enjoy!</p>Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-14557719224675319492021-03-24T11:54:00.005-06:002021-03-24T12:01:13.834-06:00What does SRE commonly do, the five principals of SRE<p> Here are some slides I've presented at a number of places people have asked me to post.</p><p><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-J0VzjR_KoUE/YFt8qrtG1AI/AAAAAAAADvA/AHIXUXOXYVk4Qee0rDpfJ0MM6wp8JpC4gCLcBGAsYHQ/s2379/Screen%2BShot%2B2021-03-24%2Bat%2B11.52.17%2BAM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1161" data-original-width="2379" height="249" src="https://1.bp.blogspot.com/-J0VzjR_KoUE/YFt8qrtG1AI/AAAAAAAADvA/AHIXUXOXYVk4Qee0rDpfJ0MM6wp8JpC4gCLcBGAsYHQ/w511-h249/Screen%2BShot%2B2021-03-24%2Bat%2B11.52.17%2BAM.png" width="511" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4R-rIPixhDTYGFJ0zrWcLC_gpttpR841el-Jske2z3F-efOn6Lh63qKvOUWuC2r2a-PSlY693aJKfBS-D1WSuOxKdUiPIH1c08Ep-OMsZ1KvAFfyJJ-mLX2248YqkNKRpSNy6mSTe27i7/s2048/Screen+Shot+2021-03-24+at+11.52.24+AM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1138" data-original-width="2048" height="284" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4R-rIPixhDTYGFJ0zrWcLC_gpttpR841el-Jske2z3F-efOn6Lh63qKvOUWuC2r2a-PSlY693aJKfBS-D1WSuOxKdUiPIH1c08Ep-OMsZ1KvAFfyJJ-mLX2248YqkNKRpSNy6mSTe27i7/w510-h284/Screen+Shot+2021-03-24+at+11.52.24+AM.png" width="510" /></a></div><br /><p>Lastly i'd like to add that "Self Correcting Systems" are vital to the success of SRE. Of course we all hear about auto-remediation or self-healing technologies. While those are self evident I personally recommend you think about your people and processes. Think about the motivations, rewards and expected human behaviors. If you focus on a target of reducing false monitoring alarms, someone MIGHT decide to just disable the alarms instead of fixing them. If you focus on auto-healing too much, you may miss the fact that most things that can/should be fixed by auto-healing is a design flaw/problem. Unfortunately we tend to ask how many fires we put out, not how many fires we prevented because "fires put out" is easier to count. We have to educate our stakeholders and leadership to learn that an ounce of prevention is worth a pound of cure!</p>Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-21118918964900295702020-09-30T16:38:00.031-06:002020-11-19T16:34:33.570-07:00What is SRE<p>SRE is an abbreviation for Service Reliability Engineering, also known as Site Reliability Engineering. SRE can also be a job description for a Service(or Site) Reliability Engineer. DevOps is a methodology (and should never be someones title) where a developer supports the code they write in production.</p><p>Service Reliability Engineering exists to improve reliability of services by writing software to keep a service functioning properly and to drive the improvement through feedback primarily around reliability, availability, observability & serviceability. SREs are also expected to do a significant amount of troubleshooting to find the root cause of an issue and not focus on the symptoms. SRE's often help Developers with Infrastructure, Deployment, Configuration, Monitoring & Metrics to make their software easy to update, manage and monitor.</p><p>SRE is a set of practices, metrics, and prescriptive ways to ensure reliability and uniformity to enable success at scale.</p><p>SRE is often a Gate to Production (grant access, make/approve/authorize production changes)</p><p>Characteristics of an SRE</p><p></p><ul style="text-align: left;"><li>Reliability</li><ul><li>Focused heavily on meeting SLO's & SLA's</li><li>Error budgets are like money, they are meant to be spent, wisely as it is limited, underspending and overspending are both bad</li><li>Embraces fact that failures will happen, plans for them</li><li>Solve production issues and restore service</li><li>Actively participate in postmortems</li><li>Has a roadmap and prioritized backlog of things to automate</li><li>SRE's is at war with </li><ul><li>toil (anything repetitive that is done by hand should be automated)</li><li>inconsistencies (snowflakes are a killer)</li><li>ignorance (visibility & data are key to managing things quickly at scale)</li></ul><li>Improves monitoring with alert correlation to reduce noise and TTR (time to repair)</li><li>SRE's mandate is to continually push for product improvements</li></ul><li>Automation/Code</li><ul><li>Minimally spends 51% of their time solving issues through code</li><li>All Incidents and Escalations should result in a runbook/workflow that eventually turns into automation (I like to call runbooks human-automation)</li><li>CI/CD Automation platforms to push out new code and fixes</li><li>Many small code pushes are far better and easier to backtrack than fewer large ones</li><li>No Production software pushes on Friday or the weekend</li><li>Knows you can't test everything, unit tests are required, but things will collide in production</li><li>Automation platform such as stackstorm to orchestrate fixes</li><li>Write tools such as auto-triage for troubleshooting (gather the logs while the engineer logs in)</li><li>Idempotent actions are your new best friend </li></ul><li>People & Process</li><ul><li>SRE's need psychological safety, one example is blameless postmortems</li><li>SRE's is a cognitively difficult task, requires minimal context switching</li><li>Participates in on-call rotation</li><li>If SRE's support a platform they do not develop entirely, they must be involved in project planning and execution with those teams.</li><li>SRE's need time to dig into incidents to not only fix, but to find root cause and take preventative measures</li></ul><li>Tooling for success</li><ul><li>SRE's need proper tooling, such as logs, time-series metrics, traces, etc.. (it's virtually impossible to understand true root cause for intermittent issues if you don't have this.)</li><li>Proper monitoring to detect failures, need to feel confident that if no alarms are triggering that the infrastructure is healthy, customers should not be the ones to tell you that you have a problem.</li><li>Never create an alarm unless it leads to an action. Warnings are useless at scale.</li></ul></ul><div>SRE Shared Vocabulary (my definitions, feel free to disagree)</div><div><ul style="text-align: left;"><li>Black Box Monitoring- All you know are the inputs and outputs, if wood goes in, and chairs comes out, thumbs up, things are good. If chairs come out broken, incomplete or not at all, you realize you have a problem.</li><li>White Box Monitoring - (I wish it was called transparent box) but this is where you know what's happening inside the machine. You see the first machine wood get cut, the second sand it, the third assemble the parts and the fourth paint the chair. You know which part of the machine is not functioning properly.</li><li>Observability - You need to make your production observable, you need to expose signals we can watch programmatically in order for the operators to understand the health of the machine.</li><li>Serviceability - How easy/difficult is it to maintain this software, to get new software into production or upgrade to a new build.</li><li>Availability - Frequency of a system to operate properly or at least within a(n) SLO/SLA</li><li>Idempotent - Something that can be applied repeatedly safely and will only make a change the first time it applies.</li><li>Immutable - Something that can't be changed after it is built/created.</li><li>Heuristic - an approach to solving a problem that uses practical methods but may not be optimal, but it will get you there.</li><li>Orthogonal - When something changes, but does so independently and does not affect the other.</li><li>Chaos Engineering - Intentionally breaking things in production to test your resiliency</li><li>Canary - Something built to test new features or configurations, or possibly a very small subset of production. This way you can find issues with limited scope of impact.</li></ul><div>Probably the biggest key to success for SRE & DevOps is the right mindset. The business must see SRE as a trusted partner who's preventing issues and keeping the service running properly. If SRE is running a production environment where they do not write all of the software, then developers sometimes think of them as operations people that they can toss software "over the fence to". While SRE is fantastic at remediating issues at scale with automation, developers need to still be accountable to the code they write and maintain responsibility for improving that code.</div></div>Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-50394974163482917752020-04-10T13:29:00.002-06:002020-04-10T13:30:42.492-06:00WD Red Price Per GB April 2020If you are like me, you do this exercise every time you need to buy a new drive.<br />
<br />
I always try to remember what it used to cost, so I'm just going to start posting them here.<br />
<br />
<table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse; width: 271px;"><colgroup><col span="2" style="width: 65pt;" width="87"></col><col style="width: 73pt;" width="97"></col></colgroup><tbody>
<tr height="21" style="height: 16pt;"><td class="xl66" colspan="3" height="21" style="border: 0.5pt solid windowtext; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; text-align: center; vertical-align: bottom; white-space: nowrap; width: 203pt;" width="271">WD RED</td></tr>
<tr height="21" style="height: 16pt;"><td class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">TB Size</td><td class="xl67" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">Price</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> Cost per GB </td></tr>
<tr height="21" style="height: 16pt;"><td align="right" class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">1</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 61.27</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 61.27</td></tr>
<tr height="21" style="height: 16pt;"><td align="right" class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">2</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 78.92</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 39.46</td></tr>
<tr height="21" style="height: 16pt;"><td align="right" class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">3</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 96.99</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 32.33</td></tr>
<tr height="21" style="height: 16pt;"><td align="right" class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">4</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 101.99</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 25.50</td></tr>
<tr height="21" style="height: 16pt;"><td align="right" class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">6</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 156.49</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 26.08</td></tr>
<tr height="21" style="height: 16pt;"><td align="right" class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">8</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 224.99</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 28.12</td></tr>
<tr height="21" style="height: 16pt;"><td align="right" class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">10</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 300.00</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 30.00</td></tr>
<tr height="21" style="height: 16pt;"><td align="right" class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">12</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 357.93</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 29.83</td></tr>
<tr height="21" style="height: 16pt;"><td align="right" class="xl67" height="21" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: 0.5pt solid windowtext; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; height: 16pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;">14</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 462.90</td><td class="xl68" style="border-bottom: 0.5pt solid windowtext; border-image: initial; border-left: none; border-right: 0.5pt solid windowtext; border-top: none; font-family: Calibri, sans-serif; font-size: 12pt; padding-left: 1px; padding-right: 1px; padding-top: 1px; vertical-align: bottom; white-space: nowrap;"> $ 33.06</td></tr>
</tbody></table>
Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-37315796386254697672019-05-03T07:52:00.000-06:002019-05-03T07:52:19.548-06:00Blogs on VMware siteRecently, most of my blogging has been directly on VMware site.<br />
<br />
I thought i'd link you to a couple of the more popular ones here.<br />
<br />
Embracing a DevOps Mindset, this is all about leading a team through a cultural transformation<br />
<a href="https://blogs.vmware.com/vov/2018/07/25/embracing-a-devops-mindset-in-vmware-it-cloud-operations/">https://blogs.vmware.com/vov/2018/07/25/embracing-a-devops-mindset-in-vmware-it-cloud-operations/</a><br />
<br />
Are we ready?, a post about how VMware makes sure its SaaS services are ready for primetime!<br />
<a href="https://blogs.vmware.com/vov/2018/12/18/9374/">https://blogs.vmware.com/vov/2018/12/18/9374/</a><br />
<br />
VMware's private cloud team Represented at VMworld.<br />
<a href="https://blogs.vmware.com/vmworld-archive-07-25-2017/2016/07/vmware-and-the-private-cloud-at-vmworld-2016.html">https://blogs.vmware.com/vmworld-archive-07-25-2017/2016/07/vmware-and-the-private-cloud-at-vmworld-2016.html</a><br />
<br />Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-91508416960089485342018-08-23T14:12:00.000-06:002019-05-03T10:52:47.920-06:00Troubleshooting 101<div style="font-family: arial, sans-serif, serif, emojifont; font-size: 14px; margin: 0px; padding: 0px;">
<style type="text/css">
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px 'Helvetica Neue'}
p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 12.0px 'Helvetica Neue'; color: #dca10d}
p.p3 {margin: 0.0px 0.0px 2.0px 0.0px; font: 14.0px 'Helvetica Neue'}
span.s1 {text-decoration: underline}
</style>
<br />
<div class="p1">
Think of yourself as a doctor, but for computers. Start with "DO NO HARM" as your credo. Don't make things worse, snapshots, GO SLOWLY, think before taking any action, ask for a double check.</div>
<div class="p1">
There are two basic approaches to troubleshooting: the stab-in-the-dark approach and the systematic approach. The stab-in-the-dark approach usually involves little knowledge of the technology involved and is completely random in nature. A systematic approach, on the other hand, involves a step-by-step approach and requires in-depth knowledge of the technology.</div>
<div class="p1">
1) When did it start? (almost always change related, planned or unplanned)</div>
<div class="p1">
Find an error message, try finding the starting time in the logs</div>
<div class="p1">
2) Isolate, isolate, isolate.</div>
<div class="p1">
How can I split this complex problem into several smaller problems. Packets go from A to Z, but don't arrive, </div>
<div class="p1">
First divide the problem in half, check if packet makes it from A-M, if it does, then check M-Z.</div>
<div class="p1">
If you see it didn't make it form M-Z, half it again, check M-T, then T-Z, then again, keep dividing in half.</div>
<div class="p1">
3) the WORST problems to troubleshoot are always two things, that agitate each other.</div>
<div class="p1">
Sometimes you have one problem, that due to redundancy, or other reasons, you don't even KNOW you have had for months.</div>
<div class="p1">
Then another thing breaks, suddenly you have a bizarre scenario that just doesn't add up.</div>
<div class="p1">
4) Check the health of EVERYTHING</div>
<div class="p1">
Log into switches, servers, (consoles people) often errors don't show up in logs, but you'll see them sitting right in of you.</div>
<div class="p1">
5) Get creative, approach the problem from different angles, ask for help, a second point of view or skillset can really help. Go play foosball, step back for 20 minutes and refresh your mind.</div>
<div class="p2">
<span class="s1"><a href="http://www.princeton.edu/~unix/Solaris/troubleshoot/methodology.html">http://www.princeton.edu/~unix/Solaris/troubleshoot/methodology.html</a></span></div>
<div class="p2">
<span class="s1"><a href="https://www.youtube.com/watch?v=PeOC16IxKwg">https://www.youtube.com/watch?v=PeOC16IxKwg</a></span></div>
<div class="p3">
<b><br />
</b></div>
<div class="p1">
More Advice:</div>
<div class="p1">
Look for workarounds, or multiple paths to restore service.</div>
<div class="p1">
If you have a known method to restore, but it may take hours or days, then try to work both paths in parallel</div>
<br /></div>
Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-85281647884766062782016-03-19T23:06:00.000-06:002016-03-19T23:24:52.083-06:00Netgear VLAN & PVID making me doubt my sanityRebuilding my home lab tonight, I got stuck because every time I plugged a cable into my switch, everything died. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-MnhUGSFfbpQ/Vu4wasdif-I/AAAAAAAABBY/l2X7SB0F9u4zGBPNK2VTE09QGK8TB9HSQ/s1600/i-feel-like-im-taking-crazy-pills.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="227" src="https://4.bp.blogspot.com/-MnhUGSFfbpQ/Vu4wasdif-I/AAAAAAAABBY/l2X7SB0F9u4zGBPNK2VTE09QGK8TB9HSQ/s320/i-feel-like-im-taking-crazy-pills.gif" width="320" /></a></div>
<br />
<br />
I came to realize that the reason for my problems was the fact I had been moving cables around in my Netgear GS748T v5 switch and even though it seemed like the VLANs configs were correct, somehow my old PVID <span style="font-size: 13.5pt; line-height: 107%;">(</span><a href="http://documentation.netgear.com/gs108t/enu/202-10337-01/GS108T_UM-06-15.html">Advanced-Port PVID Configuration</a>) settings were messing things up. The scenario I have is 4 ESX hosts, one Synology array, plus one Internet link. I have four VLANS, 1=Default/home network, 10=iSCSI, 20=Internet, 30=VSAN traffic. I just upgraded my hosts to the Intel NUC's (because I want to be like <a href="http://www.virtuallyghetto.com/2016/03/vsan-6-2-vsphere-6-0-update-2-homelab-on-6th-gen-intel-nuc.html">William Lam</a>), These <a href="http://www.intel.com/buy/us/en/product/components/intel-nuc6i5syh-498753">Intel NUC's</a> can only use the 1 onboard NIC with vSphere 6.0 U2 right now, hopefully someone will integrate a USB nic driver soon. So back to my challenge, the ESX hosts can ride on the default network and use VLAN tagging for access to the other 3 networks. My internet connection is a dumb device that can't use VLAN tagging, so I needed to find a way of integrating it. Normally that would just be an untagged port, but that doesn't work on these Netgear Switches. In order to get that to work I had to setup PVID, I used port g1 for Internet and g48 for iSCSI, and g39-42 for the ESXi hosts. The key here is that in the PVID settings, the port must be a <b><i><u>Member of the VLAN, but not Tagged.</u></i></b><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-lpnxOcC155s/Vu4tzNGEJNI/AAAAAAAABAs/A3hZnyvMFqIJUa54mDAePkyespXhIcRYQ/s1600/netgear.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://4.bp.blogspot.com/-lpnxOcC155s/Vu4tzNGEJNI/AAAAAAAABAs/A3hZnyvMFqIJUa54mDAePkyespXhIcRYQ/s400/netgear.png" width="398" /></a></div>
<br />
That seems to be working well. From the VLAN membership tab, I left my default VLAN (1) everywhere but the two untagged ports I will need my storage and internet connected to. For the other 3 VLANs I mostly emptied it out and set it up like this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-N_xt2eXDf1g/Vu4ugGSeSmI/AAAAAAAABA4/TexVVDdiivo2wDChZKcm33PcURwtIOxgg/s1600/VLAN1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="147" src="https://3.bp.blogspot.com/-N_xt2eXDf1g/Vu4ugGSeSmI/AAAAAAAABA4/TexVVDdiivo2wDChZKcm33PcURwtIOxgg/s400/VLAN1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-Io_a78zC_CU/Vu4ve_3DYRI/AAAAAAAABBM/ooBC0RaBEuww8A32tBLYw0YdMJBsLEFZA/s1600/VLAN10.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="142" src="https://2.bp.blogspot.com/-Io_a78zC_CU/Vu4ve_3DYRI/AAAAAAAABBM/ooBC0RaBEuww8A32tBLYw0YdMJBsLEFZA/s400/VLAN10.png" width="400" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkl_w8yJjHAkD5d5s7-Lzy6jSM89vgQUws8t1n-xtod6Sp1RcXsVYUMnCxuM85sV-9B4H-4uooLpLjPbOi-qxDmfdU7Avy-GsMf5CkmGJl9Y8Go3Ju6EiWhzEGpTO6PSRkdkg5fiblySwo/s1600/VLAN.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="155" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkl_w8yJjHAkD5d5s7-Lzy6jSM89vgQUws8t1n-xtod6Sp1RcXsVYUMnCxuM85sV-9B4H-4uooLpLjPbOi-qxDmfdU7Avy-GsMf5CkmGJl9Y8Go3Ju6EiWhzEGpTO6PSRkdkg5fiblySwo/s400/VLAN.png" width="400" /></a><a href="https://1.bp.blogspot.com/-YYJe8do7L7A/Vu4vexsQTTI/AAAAAAAABBI/KvSkhWp30Ps0I5qxWU1Kjyr31ZFRm-4Rw/s1600/VLAN30.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="145" src="https://1.bp.blogspot.com/-YYJe8do7L7A/Vu4vexsQTTI/AAAAAAAABBI/KvSkhWp30Ps0I5qxWU1Kjyr31ZFRm-4Rw/s400/VLAN30.png" width="400" /></a></div>
<br />
If you have a similar setup and you get stuck, I hope this helps you!<br />
<br />Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com1tag:blogger.com,1999:blog-3991424965072200600.post-70124850068968346482013-12-09T09:30:00.001-07:002013-12-09T09:36:45.824-07:00VMware vSAN IOPS testing<p>Take this with a grain of sand, these are only initial figures. I am using a combination of IOMeter for Windows and fio for Linux.</p> <p>Baseline redundancy and caching, no storage profiles used, only using vSAN as a datastore (I’ll do the other options later)</p> <p>My vSAN is made of 3 identical ESXi hosts, with a single SSD Samsung 840 250GB, and two Seagate 750GB SATA drives. vSAN has a dedicated single 1GB connection, no jumbo frames used. (yes there could be bottlenecks at several spots, I haven’t dug that deeply, this is just a ‘first pass’ test)</p> <p>The end result of this VERY BASIC test is this:</p> <p><em>vSAN random reads were an average of <strong><u>31 times faster</u></strong> than a single SATA disk</em></p> <p><em>vSAN random writes were an average <strong><u>9.1 times faster</u></strong> than a single SATA disk</em></p> <p> </p> <p>More Details Below:</p> <p>Regular single disk performance (just for a baseline before I begin vSAN testing)</p> <p>Random Read (16k block size)</p> <p>first test = 79 IOPS</p> <p>second test = 79 IOPS</p> <p>Random Write (16k block size)</p> <p>first test = 127 IOPS</p> <p>second test = 123 IOPS</p> <p>vSAN disk performance with same VM vMotion to the vSAN </p> <p>Random Read (16k block size)</p> <p>first test = 2440 IOPS </p> <p>second test = 2472 IOPS</p> <p>Random Write (16k block size)</p> <p>first test 1126 IOPS </p> <p>second test 1158 IOPS</p> <p>Commands used in fio:</p> <p>sudo fio --directory=/mnt/volume --name fio_test --direct=1 --rw=randread --bs=16k --size=1G --numjobs=3 --time_based --runtime=120 —-group_reporting <p>sudo fio --directory=/mnt/volume --name fio_test --direct=1 --rw=randwrite --bs=16k --size=1G --numjobs=3 --time_based --runtime=120 —-group_reporting <p>I mentioned I did use IOMeter in windows, the initial results were very similar to the fio results above. I will post those once I have the time try each solution and go deeper into identifying bottlenecks and getting more detailed, adding more hosts, etc…</p> Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com2tag:blogger.com,1999:blog-3991424965072200600.post-56425786126598833482013-12-08T17:36:00.001-07:002013-12-09T09:11:15.906-07:00VMware vSphere 5.5 vSAN beta ineligable disks<div style="text-align: center"> <div style="text-align: left">While building my home lab to use vSAN and NSX following <a href="https://twitter.com/VMwareStorage">Cormac Hogans</a>'s great <a href="http://www.vmware.com/files/pdf/products/vsan/POC_Guide_VSAN.pdf">instructions</a>, I've encountered an issue that the disk I am trying to use for vSAN are not showing as available. In the "Cluster/Manage/Virtual SAN/Disk Management" under Disk Groups, I see only one of my 3 hosts has 0/2 disks in use, the others show 0/1. My setup is this, I purchased 3 new 250GB Samsung SSD drives (one for each host), and am trying to re-use 6 older Seagate 750GB SATA drives. My first thought, is why does it only say 0/1 in use on two of the servers? I have 4 drives in that server, a 60GB boot drive, 1 SSD, & 2 SATA drives, so why doesn't it say 0/3 or 0/4? I noticed in the bottom pane, I can choose to show ineligable drives, there I see the 3 drives I can't use. I understand why I can't use my Toshiba boot drive, but why do my 750GB Seagate drives also show Ineligable?<br><br><a href="http://www.stopsmokingcrack.com/Blog/ineligable_vsan.png"><img border="0" src="http://www.stopsmokingcrack.com/Blog/ineligable_vsan.png" width="400" height="210"></a><br><br>I played with enabling AHCI, but knowing there is a bug in the beta I wanted to avoid it. See here: <a href="http://blogs.vmware.com/vsphere/2013/09/vsan-and-storage-controllers.html">http://blogs.vmware.com/vsphere/2013/09/vsan-and-storage-controllers.html</a>. This unfortunately did not change the situation. I finally realized that possibly those drives still had a legacy partition on them. After nuking the partitions on those drives, the disk now show up as eligable drives. I tried this first on my server smblab2, and you see that 0/3 are not in use, which is what I would have expected originally. Not in use in this context basically means "eligable".<br><br><a href="http://www.stopsmokingcrack.com/Blog/eligable_vsan.png"><img border="0" src="http://www.stopsmokingcrack.com/Blog/eligable_vsan.png" width="400" height="210"></a><br>I was then able to Claim the disks for VSAN Use:<br><br><a href="http://www.stopsmokingcrack.com/Blog/vsan_Healthy_disks.png"><img border="0" src="http://www.stopsmokingcrack.com/Blog/vsan_Healthy_disks.png" width="400" height="209"></a><br>Then finally create the disk groups.<br><br><a href="http://www.stopsmokingcrack.com/Blog/Disk_groups_created.png"><img border="0" src="http://www.stopsmokingcrack.com/Blog/Disk_groups_created.png" width="400" height="277"></a><br>Many others suggest running vSAN in a Virtual environment, which is great for learning, you can even get the experience doing the Hands on Labs (<a href="http://labs.hol.vmware.com/">Free 24/7 Now!</a>), but I wanted to do some performance testing, and for that I needed a physical environment. Now that I've gotten past my little problem, it's working great!</div></div> Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-85183162273121412702013-11-25T10:59:00.001-07:002013-11-25T11:35:51.303-07:00VMworld 2013 Hands On Labs Dashboards<p>I’ve been asked several times to publish these as not everyone got to take pictures, or they were not clear enough. </p> <p>We chose to build custom <a href="http://www.vmware.com/products/vcenter-operations-management/">VMware® vCenter™ Operations Management Suite™</a> (vC Ops) dashboards. The Built-in vC Ops dashboards are build around a normal datacenter where workloads live indefinitely, and trending is key, for our environment, workloads are created and destroyed so frequently, that this data isn’t key. Also in a normal environment, the VM’s are crucial, but in ours, the infrastructure is.</p> <p>HOL was built with two major sites for each show. For the EMEA VMworld, we used London & Las Vegas. The dashboards below were taken right before the show opened in the morning, so there isn’t much if any load in London, there is some load in Las Vegas because that is where we were running the 24/7 public Hands on Labs. The first dashboard for each site contains metrics around traditional constraints, such as CPU, Memory, Storage IOPS, Storage Usage, & Network Bandwidth. These are all done at the vCenter level as the lab VM’s only live 90 minutes we really don’t care much about their individual performance as we can’t tune them before they are recycled. We do care about the underlying infrastructure and we are watching to make sure they have plenty of every resource so that they can run optimally. Much of the data that we fed into vC Ops comes from <a href="http://www.vmware.com/products/vfabric-hyperic/">vCenter Hyperic</a></p> <p><a href="http://www.stopsmokingcrack.com/Blog/london1.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="London1" border="0" alt="London1" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvWJd_CkdaM9txzcEc_8gY8NWLMVgD_vuHEoF0YpqDfUFdy3NdDWr4dYac-pTGA_oTuO4ZxqASe_VPdYYx5oUmpqi7aNB3ODnL4FVXCh03wVbas87PY9-4Kcz_4MW86riqY3IpEuql31vZ/?imgmax=800" width="520" height="800" ?></a></p> <p>The second dashboard below is looking at vCloud Director Application performance. We looked directly into inspecting each Cell Server for # of proxy connections, cpu, & memory. We also looked into the vSM to verify the health of the vShield Manager VM’s. Lastly we were concerned with the SQL DB performance, so we were watching the transactional performance, making sure there wasn’t too many waiting tasks, or DB wait times.</p> <p><a href="http://www.stopsmokingcrack.com/Blog/london2.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="London2" border="0" alt="London2" src="http://lh3.ggpht.com/-Jr2684cV8ec/UpOPzjq5HDI/AAAAAAAAA9c/3w9kHy9vtOo/London2%25255B8%25255D.png?imgmax=800" width="520" height="808"></a></p> <p>We also leveraged <a href="http://www.vmware.com/products/vcenter-log-insight/">VMware vCenter Log Insight</a> to consolidate our log views. This was very helpful for troubleshooting to be able to trace something throughout the stack. We also leveraged the alerting functionality to email us when known errors strings occurred in the logs so that we could be on top of any issue before users noticed.</p> <p><a href="http://www.stopsmokingcrack.com/Blog/london3.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="london3" border="0" alt="london3" src="http://lh5.ggpht.com/-y1mAGDE5Giw/UpOP0q17kSI/AAAAAAAAA9k/7HJAMokobGQ/london3%25255B7%25255D.png?imgmax=800" width="520" height="808"></a></p> <p>Same as Screen #1 above, just for Las Vegas, again you notice more boxes, that is because it is twice the size. The London facility only ran the show, the Las Vegas DC below ran both the show and the public 24/7 Hands on Labs.</p> <p><a href="http://www.stopsmokingcrack.com/Blog/vegas1.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="vegas1" border="0" alt="vegas1" src="http://lh4.ggpht.com/-4txNLM8ZLAk/UpOP1tbSOpI/AAAAAAAAA9s/qX8ZEtCvNMg/vegas1%25255B6%25255D.png?imgmax=800" width="520" height="808"></a></p> <p>Same as #2 Above.</p> <p><a href="http://www.stopsmokingcrack.com/Blog/vegas2.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="vegas2" border="0" alt="vegas2" src="http://lh6.ggpht.com/-aVwRDq-TKSI/UpOP21jjm5I/AAAAAAAAA90/Ryut7D_QKEg/vegas2%25255B6%25255D.png?imgmax=800" width="520" height="808"></a></p> <p>Same as #3 above, except that we show you the custom dashboard we created with <a href="http://www.vmware.com/products/vcenter-log-insight/">VMware vCenter Log Insight</a>, so that we could see trends of errors, this was very helpful to see when errors happen that we might otherwise not be looking for.</p> <p><a href="http://www.stopsmokingcrack.com/Blog/vegas3.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="vegas3" border="0" alt="vegas3" src="http://lh3.ggpht.com/-H_mkCLY9pMQ/UpOP_7EM9WI/AAAAAAAAA98/-rOkK-RXmXs/vegas3%25255B8%25255D.png?imgmax=800" width="520" height="808"></a></p> <p>The final dashboard below is to watch the <a href="http://www.emc.com/storage/xtremio/index.htm">EMC XtremIO</a> performance. These bricks had amazing performance and were able to handle any load we threw at it. With the inline deduplication we were able to use only a few TB of real flash storage to provide 100’s of TB of allocated storage. Matt Cowger from EMC did a great <a href="http://blog.cowger.us/?p=388">blog post</a> about our usage </p> <p><a href="http://www.stopsmokingcrack.com/Blog/xio.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="xio" border="0" alt="xio" src="http://lh6.ggpht.com/-VqJ9LosFAcg/UpOQA8cjLbI/AAAAAAAAA9I/QadZMlpKCgQ/xio%25255B6%25255D.png?imgmax=800" width="520" height="800"></a></p> <p>Final Numbers:</p> <p>HOL US served 9,597 Labs with 85,873 VM’s</p> <p>HOL EMEA served 3,217 Labs with 36,305 VM’s.</p> <p>We achieved a nearly perfect uptime. We did have a physical blade failure, but HA kicked in and did it’s job, we also had a couple hard drive failures, once again a hot spare took over and automatically resolved the issue. During both occurrences, we saw a red spike from the vC Ops dashboards, we observed the issue, but did not need to make any changes, we just watched the technology magically self-heal as it’s supposed to.</p> Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com1tag:blogger.com,1999:blog-3991424965072200600.post-69345600093074800692013-08-28T10:51:00.000-06:002013-08-28T11:06:32.504-06:00VMworld HOL using VCVA (vCenter Virtual Appliance)This is the first of a series of HOL posts about "how we did it".<br />
<br />
For the primary workload, we used the vCenter Virtual Appliance using the local Postgres database. <br />
<br />
Due to the unusually high churn rate of HOL, we need to have a high ratio of vCenters. These vCenters needed to have a lot of horsepower behind them to survive this churn.<br />
<br />
1) Paravirtualized SCSI adaptors for disk controllers for the VCVA vm.<br />
2) Created 2 additional dedicated datastores (Luns) for each of the DB & Logs on the VCVA vm.<br />
3) 4 CPU's x 32GB memory (we might have gone a bit high on memory)<br />
4) Removed all long term logging and rollups, we are doing all stats in vC Ops.<br />
5) Increased heap sizes to large for the SPS, tomcat inventory & vCenter process.<br />
<br />
The only downside to the VCVA is the fact that it doesn't support linked mode, but you can get around that with the NGC & SSO. http://www.virtuallyghetto.com/2012/09/automatically-join-multiple-vcsa-51.htmlBrian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-6351419561554451082013-08-28T10:23:00.000-06:002013-08-28T10:55:17.932-06:00 ESXi 5.1vHost vDS ports on an stateless reverts to 512 after reboot<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="font-family: 'Times New Roman', serif; font-size: 13.5pt;">By default when you set
the ports on hosts max to 1024, after reboot, it goes back to 512 on a stateless host.
This is a known issue in the <a href="http://www.vmware.com/support/vsphere5/doc/vsphere-esx-vcenter-server-51-release-notes.html">5.1 release notes</a>.</span><span style="color: black; text-decoration: none; text-underline: none;"></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
</div>
<ul>
<li><span style="font-family: 'Times New Roman', serif; font-size: 13.5pt;">maxProxySwitchPorts setting
not persistent after stateless host reboot </span></li>
<li><span style="font-family: 'Times New Roman', serif; font-size: 13.5pt;">The maximum number of ports on a host is reset to 512 after the host is
rebooted and a host profile applied. When you
set maxProxySwitchPorts on a specific stateless host on a distributed
switch, the setting might not persist when the host is rebooted. This applies
only to stateless hosts that are part of a distributed switch and have had
themaxProxySwitchPorts setting changed.</span></li>
<li><span style="font-family: 'Times New Roman', serif; font-size: 13.5pt;">Workaround: Manually
change the maxProxySwitchPorts settings for the hosts after reboot.</span></li>
</ul>
<br />
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="font-family: 'Times New Roman', serif; font-size: 13.5pt;">There are 3 ways to make
this change, i'll discuss them here.<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="font-family: 'Times New Roman', serif; font-size: 13.5pt;">1) vSphere Windows
Client, this way seems to work, but does not. The UI states that the host
must be rebooted after the setting is changed. Some experimenting looked like
the change did take effect immediately without a reboot and I've confirm in the
API guide that a reboot is no longer needed for ESXi 5.1 hosts (<span style="color: black; text-decoration: none; text-underline: none;"><a href="http://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.wssdk.apiref.doc%2Fvim.dvs.HostMember.ConfigSpec.htm">http://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.wssdk.apiref.doc%2Fvim.dvs.HostMember.ConfigSpec.htm</a>l</span><span id="goog_250324438"></span><span id="goog_250324439"></span><a href="http://www.blogger.com/"></a>),
so the "must reboot" label just a UI artifact.<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="font-family: 'Times New Roman', serif; font-size: 13.5pt;">2) Workaround stated
above in the release notes using PowerCLI, While changing the
maxProxySwitchPorts with PowerCLI does work, it's a pain. <o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="font-family: 'Times New Roman', serif; font-size: 13.5pt;">3) Using the NGC (next
gen web client). We found the workaround to be setting the "Default
max number of ports per host" via the NGC and this does persist between
reboots. We tested this on a host that we rebooted and it did come up with 1024
ports. <o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-raM08aiy4qI/Uh4Zq6SBrAI/AAAAAAAAA6I/Tv868j9uW4c/s1600/IMG_14082013_174800.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="http://1.bp.blogspot.com/-raM08aiy4qI/Uh4Zq6SBrAI/AAAAAAAAA6I/Tv868j9uW4c/s640/IMG_14082013_174800.png" width="640" /></a></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<span style="font-family: 'Times New Roman', serif; font-size: 13.5pt;">The credit for this goes
to my members, <a href="https://twitter.com/jacobmross">Jacob Ross</a> & <a href="https://twitter.com/humannumber1">Joe Keegan</a>.<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
</div>
<div class="MsoNormal">
<br /></div>
Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-12641607609163551202013-07-03T09:40:00.001-06:002013-07-03T09:44:54.086-06:00vSphere 5.1 Update1 PSOD Fix Build<p>VMware released 5.1U1 on April 25th with Build 1065491. There were some critical bugs identified and fixed, so if you are using 5.1U1 with Intel Processors, you may want to use build 1117900, which came out May 22nd. This build fixes some bugs about occasional PSOD (purple screen of death) related Flex Priority in Intel processors (part of the VT featureset). </p> <p>Here is the Build KB <a href="http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2051207">http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2051207</a></p> <p>Download the Build <a href="http://www.vmware.com/patchmgr/findPatch.portal?product=ESXi+(Embedded+and+Installable)&version=121">http://www.vmware.com/patchmgr/findPatch.portal?product=ESXi+(Embedded+and+Installable)&version=121</a></p> Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-55843830144417770442013-07-02T15:06:00.001-06:002013-07-02T15:06:07.001-06:00Storage IOPS Planning<p> </p> <p>The limit on the VNX is the Storage Processors. Utilizing the aggressive numbers is likely to see significant impact to the workloads.</p> <table border="0" cellspacing="0" cellpadding="2" width="400"> <tbody> <tr> <td valign="top" width="100">Array</td> <td valign="top" width="100">Conservative</td> <td valign="top" width="100">Typical</td> <td valign="top" width="100">Aggressive</td></tr> <tr> <td valign="top" width="100">VNX7500</td> <td valign="top" width="100">60,000</td> <td valign="top" width="100">80,0000</td> <td valign="top" width="100">100,000</td></tr> <tr> <td valign="top" width="100">S200 Isilon</td> <td valign="top" width="100">8,000 (per node)</td> <td valign="top" width="100">10,000</td> <td valign="top" width="100">12,000</td></tr></tbody></table> Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com3tag:blogger.com,1999:blog-3991424965072200600.post-84499824021926081132013-05-09T12:13:00.003-06:002013-05-09T12:25:54.569-06:00New performance and optimizations guides for Cloud and vSpherevCloud <br />
<span style="color: black; font-family: 'Calibri','sans-serif'; font-size: 11pt; mso-ansi-language: EN-US; mso-bidi-language: AR-SA; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin;"><a href="http://www.vmware.com/files/pdf/techpaper/VMware-vCloud-Director51-Perf.pdf" target="_blank"><span style="color: blue;">http://www.vmware.com/files/pdf/techpaper/VMware-vCloud-Director51-Perf.pdf</span></a></span><br />vSphere<br />
<span style="color: black; font-family: 'Calibri','sans-serif'; font-size: 11pt; mso-ansi-language: EN-US; mso-bidi-language: AR-SA; mso-fareast-font-family: Calibri; mso-fareast-language: EN-US; mso-fareast-theme-font: minor-latin;"><span style="color: black; font-family: 'Calibri','sans-serif'; font-size: 11pt; mso-ansi-language: EN-US; mso-bidi-language: AR-SA; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-US;"><a href="http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.1.pdf"><span style="color: blue;">http://www.vmware.com/pdf/Perf_Best_Practices_vSphere5.1.pdf</span></a></span></span><br />
<br />Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-47213318803907147842013-05-09T11:46:00.000-06:002013-05-09T12:27:37.186-06:00Getting your memory savings back on with ESXi and small memory pagesMany other VCDX's have posted all the technical details about this before, so this isn't revolutionary, but back in the day's of 32 bit OS's their memory was stored in small 4k pages on the ESXi host. TPS was great at comparing and colapsing these and generating huge memory savings, typically about 30%. Now with modern 64 bit OS's, this memory is stored in large 2MB pages by default. The reason for the large pages is some performance enhancement, and i'm sure in a highly CPU/Memory latency sensitive environment there is some benefit. However, in every environmnet I've ever worked in, the bottleneck is qty of memory and disk I/O. TPS does begin to break down the large pages into smaller pages when the system is in the last 6% of memory available, before it starts to swap, this is great, but usually, it's too late to really matter. I recommend disabling the large pages, and therefore having ESXi store it in the smaller blocks. You can force this to happen by changing a property on ESXi on the host under advanced "mem" called Mem.AllocGuestLargePage and change it from 1 = large pages enabled to 0 = large pages disabled. After you reboot the host, your VM's should begin to use the smaller pages and almost immediately begin saving your memory with TPS.<o:p></o:p>Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-6820293944425867602013-04-26T15:04:00.001-06:002013-04-26T15:05:10.894-06:00vDS port limitsMnaging the maximum number of ports you can have on a distributed switch has always been difficult for those of us running large clouds. It appears that with vSphere 5.1 that limit has mostly been removed. In vSphere 4.0 the limit was 4,096, in 4.1, it was 8,192 but could be modified with PowerCLI to 20,000. As of vSphere 5.0 the limit was 30k, now in 5.1 that limit is 60k. However today we discovered the limit in the database is actually 2,147,483,647 (2.1Billion). I guess I won't have to worry about those errors anymore...Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-34554784303729010462013-03-11T16:09:00.002-06:002013-03-11T16:10:24.601-06:00Datastore connectivity issuesWe have a shared datastore to provide global catalogs to our various ORG's in the vCloud. For whatever reason several ESX hosts across mulitple clusters were reporting APD (all path's down) when trying to connect to this VNX NFS Export. After putting those hosts into maintenance mode, everything seemed happy again. We did quite a bit of digging, it turned out all servers had the same patch level, the NFS export seemed properly configured, we were stumped. Looking on the Nexus 7k, we saw that one of the ports in the VNX portchannel had a low Rx Power -11.30 dBm, and 75414 CRC errors on RX. Once we replaced the SFP & Fiber Optic Cable, everything is happy again.Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com0tag:blogger.com,1999:blog-3991424965072200600.post-32986440083060069812012-09-20T10:47:00.001-06:002012-09-20T10:48:53.210-06:00Can’t create a VMware Standard Switch vSS with vSphere web (next gen) client<p>I thought perhaps we had removed this functionality in vSphere 5.1, but it is still there, just very buried.  You have to go into the properties of the Host, Actions/All vCenter Actions/ Add Networking.. (see below)</p> <p><a href="http://lh3.ggpht.com/-bPZRmUutDbc/UFtIsKOvq1I/AAAAAAAAA3Q/Tx6bB95PNrw/s1600-h/image%25255B13%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-2UVo_0-M_NE/UFtItOfYVrI/AAAAAAAAA3U/cU_SvMKoAdc/image_thumb%25255B7%25255D.png?imgmax=800" width="545" height="511" /></a></p> <p>Here is the published <a href="http://pubs.vmware.com/vsphere-51/index.jsp?topic=%2Fcom.vmware.vsphere.networking.doc%2FGUID-1C0D8D8D-F9A5-4443-9AE7-544742630D39.html" target="_blank">document</a> on how to do it.</p> Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com1tag:blogger.com,1999:blog-3991424965072200600.post-81488859362570747582012-09-19T13:34:00.001-06:002012-09-19T13:45:41.126-06:00Deploying a VMware vCloud Director (vCD) 5.1 virtual appliance with MS SQL backend<p>This is a guide for deploying vCloud in a LAB environment, these settings are not the most secure or performant, but this should get you up and running with vCloud 5.1 so you can test and learn it. The easiest way is with the appliance, you don’t need to use a MS SQL DB, but occasionally I need to crack open the DB and I am the most comfortable with that technology.</p> <ol> <li>My Assumptions about what you already have: <ol> <li>One ESXi Host with the following VM’s on it. <ol> <li>Windows with MS SQL DB (I’m using MS SQL 2008 R2) </li> <li>vShield Manager 5.1 with an IP set (also known as vCloud Networking and Security 5.1) </li> <li>vSphere 5.1 vCenter (can be the appliance) </li> <li>Available resources to Deploy vCloud Director Appliance </li> </ol> </li> <li>VCP or equivalent level of knowledge </li> </ol> </li> <li>Prepare your Database (same steps as with non-appliance)  <ol> <li>Again, I am assuming you have MS SQL 2008R2 installed, without a local firewall, or ports opened. </li> <li><a href="http://pubs.vmware.com/vcd-51/index.jsp?topic=%2Fcom.vmware.vcloud.install.doc_51%2FGUID-A3CDF724-7BFA-4BD0-95C4-55AC7A9F4055.html">This</a> is a great article, follow it, I will paste the highlights from it below, you can copy/paste these commands into SQL Query analyzer!! </li> </ol> <p>1)    Configure the database server. <br />A database server configured with 16GB of memory, 100GB storage, and 4 CPUs should be adequate for most vCloud Director clusters. (this is for production level quality) <br />2)    Specify Mixed Mode authentication during SQL Server setup. <br />Windows Authentication is not supported when using SQL Server with vCloud Director. <br />3)    Create the database instance. <br />The following script creates the database and log files, specifying the proper collation sequence.</p> <p>USE [master] <br />GO <br />CREATE DATABASE [vcloud] ON PRIMARY <br />(NAME = N'vcloud', FILENAME = N'C:\vcloud.mdf', SIZE = 100MB, FILEGROWTH = 10% ) <br />LOG ON <br />(NAME = N'vcdb_log', FILENAME = N'C:\vcloud.ldf', SIZE = 1MB, FILEGROWTH = 10%) <br />COLLATE Latin1_General_CS_AS <br />GO</p> <p>The values shown for SIZE are suggestions. You might need to use larger values. <br />4)    Set the transaction isolation level. <br />The following script sets the database isolation level to READ_COMMITTED_SNAPSHOT.</p> <p>USE [vcloud] <br />GO <br />ALTER DATABASE [vcloud] SET SINGLE_USER WITH ROLLBACK IMMEDIATE; <br />ALTER DATABASE [vcloud] SET ALLOW_SNAPSHOT_ISOLATION ON; <br />ALTER DATABASE [vcloud] SET READ_COMMITTED_SNAPSHOT ON WITH NO_WAIT; <br />ALTER DATABASE [vcloud] SET MULTI_USER; <br />GO</p> <p>For more about transaction isolation, see <a href="http://msdn.microsoft.com/en-us/library/ms173763.aspx">http://msdn.microsoft.com/en-us/library/ms173763.aspx</a>.</p> <p>5)    Create the vCloud Director database user account. <br />The following script creates database user name vcloud with password vcloudpass.</p> <p>USE [vcloud] <br />GO <br />CREATE LOGIN [vcloud] WITH PASSWORD = 'vcloudpass', DEFAULT_DATABASE =[vcloud], <br />   DEFAULT_LANGUAGE =[us_english], CHECK_POLICY=OFF <br />GO <br />CREATE USER [vcloud] for LOGIN [vcloud] <br />GO</p> <p>6)    Assign permissions to the vCloud Director database user account. <br />The following script assigns the db_owner role to the database user created in Step 5.</p> <p>USE [vcloud] <br />GO <br />sp_addrolemember [db_owner], [vcloud] <br />GO</p> <p> </p> </li> <li>Deploy and configure vCloud virtual appliance <ol> <li>Login to vCenter with vSphere client </li> <li>Click on File/Deploy OVF template, choose vCloud-Director-VA-T2-5.1.0.0-817173_OVF10.ova </li> <li>After you choose the obvious options, you should get a properties page to fill out like below </li> <li><a href="http://lh3.ggpht.com/-zJ-VuzTidhQ/UFoeOIL4YxI/AAAAAAAAA18/B68-zDj5i9U/s1600-h/image%25255B3%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOiynzKyqV8YZQfK3lxyo4_hGzgXiSRuvfybV6ccVPNvcIRZblf8u_J-posXFtxrFxWzvjMe3rJR2KzqIND654dsmju6wjQbbDQan2M1sm1GualI2FLPgtt9LHiVxikePOA7WAoHVVuME9/?imgmax=800" width="404" height="391" /></a> </li> <li><a href="http://lh5.ggpht.com/-TNoWjs-TnQs/UFoePV_xDWI/AAAAAAAAA2M/UyQBfcDLjoo/s1600-h/image%25255B11%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-3KCrYAM5gSc/UFoeQHQqYaI/AAAAAAAAA2U/oRQnUXPFzas/image_thumb%25255B5%25255D.png?imgmax=800" width="409" height="396" /></a> </li> <li>Fill in the options </li> <li>Scroll Down, Fill in Database Name, vcloud if you used my info above </li> <li>Fill out Networking Properties (or leave blank for DHCP) </li> <li>After the standard deploy progress bar, you will see a long delay(5-10 minutes) during VM boot up while it sets up the DB, it’s not hung, just give it time.  It will occasionally ask questions, don’t answer them, just let it go<strong><em>..go get coffee..</em></strong> </li> <li>Eventually you should see a screen like this indicating that installation is finally complete: </li> <li><a href="http://lh3.ggpht.com/-WaMjtDj0Axg/UFoeQ1Hl2oI/AAAAAAAAA2c/2g2NJcRWEFQ/s1600-h/image%25255B19%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-MrMsJsU08So/UFoeRTL7CCI/AAAAAAAAA2k/bojdgkly6c4/image_thumb%25255B9%25255D.png?imgmax=800" width="407" height="236" /></a> </li> <li>Login to <a href="https://ipaddress:5480">https://ipaddress:5480</a> admin/vmware if you need to configure the VM any further </li> <li>Login to <a href="https://ipaddress/cloud/">https://ipaddress/cloud/</a> to begin configuring vCloud, but that will be my next blog post.(link soon) </li> </ol> </li> </ol> <p>Good Links:</p> <h3><a href="https://www.vmware.com/support/vcd/doc/rel_notes_vcloud_director_51.html" target="_blank"><font size="2"><font style="font-weight: normal">vCloud Director 5.1 Release Notes</font></font></a></h3> <p><a href="http://pubs.vmware.com/vcd-51/index.jsp?topic=%2Fcom.vmware.vcloud.install.doc_51%2FGUID-F14315CC-B373-4A21-A3D9-270FFCF0A417.html" target="_blank">VMware vCloud Director 5.1 Documentation Center</a></p> <p><a href="https://www.vmware.com/support/pubs/vcd_pubs.html" target="_blank">VMware vCloud Directory Documentation</a></p> Brian Smithhttp://www.blogger.com/profile/01031494739091598571noreply@blogger.com2