The Renegade Coder https://therenegadecoder.com Code First. Ask Questions Later. Mon, 14 Oct 2019 17:21:05 +0000 en-US hourly 1 https://wordpress.org/?v=5.2.4 https://i0.wp.com/therenegadecoder.com/wp-content/uploads/2017/05/the-renegade-coder-icon-cropped.png?fit=32%2C32&ssl=1 The Renegade Coder https://therenegadecoder.com 32 32 127809749 My WordPress Website Is Really, Really Slow https://therenegadecoder.com/meta/my-wordpress-website-is-really-really-slow/ https://therenegadecoder.com/meta/my-wordpress-website-is-really-really-slow/#respond Mon, 14 Oct 2019 14:00:30 +0000 https://therenegadecoder.com/?p=18754

Is your WordPress website really slow? Same! Luckily, I found a few ways to fix it. It just took a little bit of research.

The post My WordPress Website Is Really, Really Slow appeared first on The Renegade Coder.

]]>

As someone who has been blogging with WordPress for a little while now, I was starting to wonder if I was ever going to be able to speed up my site. After all, I didn’t personally think it was all that slow, but various page testing tools told me otherwise. Naturally, I felt like it had to be affecting my page rankings, so I decided to accept the fact that my WordPress website is slow and take matters into my own hands.

After improving my site speed, I was able to come up with three tips. First, get an idea of where your site is at right now using some performance auditing tool. Then, take a good look at your theme and plugins in isolation. If all else fails, disable advertisements—they are slow!

If you’re interested in the entire process I went through including performance screenshots, check out the remainder of this article. Otherwise, jump straight to the Lessons Learned section for some quick tips.

Table of Contents

Inspiration

Right now, I’m nearing my 3rd year of blogging (started in December of 2016), and I’m starting to wonder what I can do to improve my results. While I’m not exactly upset with my progress, I do think it could be better.

As I try to dig into what I’ve been doing wrong, there’s a lot I can point to:

  • Not focusing enough on a niche
  • Writing content that no one cares about
  • Not writing evergreen content
  • Not focusing on speed

And, there it is! One of my biggest hurdles with this site is speed. After all, if you couldn’t tell, my site is really, really slow.

How Slow is Slow?

Depending on where you open my website, you might not notice any issues. For instance, the only issues I really noticed were related to asynchronous resources not loading right away. In other words, it’s not like you couldn’t read my articles; you’d just be waiting for those resources for a few seconds.

That said, if you were to run any performance testing software on my site. you would have gotten some very different results. Naturally, I want to share some of those results with you. As it turns out, my WordPress website is slow.

Methodology

Throughout this article, I will refer to a lot of different performance metrics, so I want to be sure that I lay out exactly how testing was performed first.

To test my website, I used a combination of Incognito Mode and Google’s auditing tool, Lighthouse. That way, my performance tests were all handled through Chrome DevTools without any interference from plugins.

When using Lighthouse, there are several settings you can choose. Personally, I stuck with all the default settings:

  • Device: Desktop
  • Audits
    • Performance (check)
    • Progressive Web App (check)
    • Best practices (check)
    • Accessibility (check)
    • SEO (check)
  • Throttling: Simulated Slow 4G, 4x CPU Slowdown
  • Clear Storage (check)

Then, after execution, you’ll get a breakdown of all the audits above with scores between 0 and 100. From there, the tool attempts to provide additional feedback like diagnostics and opportunities, so you can find ways to improve your site.

In addition, I used the Health Check & Troubleshooting plugin to isolate problem plugins. For more information, check out my article on WordPress cleanup.

Overall, the testing you’ll see throughout the remainder of this article took me about two days. Hopefully, it won’t take you nearly as long.

Homepage Speed

If you run Google Chrome’s auditing tools on my homepage, you’ll get the results below. In particular, the SEO is fine, but the performance is atrocious.

Initial WordPress Homepage Speed Test
Performance test on my current homepage.

When you dig into why the performance is so bad, you’ll find the following:

  • First Contentful Paint: 3.0 s
  • Speed Index: 14.1 s
  • Time to Interactive: 17.7 s
  • First Meaningful Paint: 4.8 s
  • First CPU Idle: 16.4 s
  • Maximum Potential First Input Delay: 3.090 ms

Then, apparently, I’m able to fix these times by addressing the following issues:

  • Properly Size Images: 2.8 s
  • Eliminate Render-blocking Resources: 0.99 s
  • Preconnect to Required Origins: 0.53 s

Of course, this is the homepage, so does performance really matter? After all, I don’t think people are finding me through my homepage. Instead, they’re hitting specific articles, so I figured we should test one of those.

Article Performance

When I ran the performance experiment on one of my latest articles, I found that the performance was just as terrible.

Initial WordPress Article Speed Test
Performance test on my The Lisp Program Language article.

In particular, the tool mentioned the following:

  • First Contentful Paint: 2.9 s
  • Speed Index: 9.4 s
  • Time to Interactive: 18.3 s
  • First Meaningful Paint: 6.6 s
  • First CPU Idle: 17.4 s
  • Max Potential First Input Delay: 1,950 ms

In addition, the tool mentions I can do the following to fix up the site:

  • Eliminate Render-blocking Resources: 0.77 s
  • Preconnect to Required Origins: 0.5 s
  • Removed Unused CSS: 0.42 s
  • Defer Offscreen Images: 0.15 s

Clearly, after peaking at a couple of pages, there are some systematic issues on my site.

Systematic Issues

If my site is slow across the board, I need to start looking at systematic errors. To start, I decided to ask the following questions:

  • Is my theme at fault?
  • Are my plugins at fault?
  • Is my host at fault?

To me, these are the three main sources of systematic error. As a result, I decided to dig into the first two.

Control

Before I ran any tests, I decided to perform a control test. To do that, I used the health check plugin to run my site in troubleshoot mode. Then, I ran the same performance test from above.

Control WordPress Homepage Performance Test
Performance test on homepage—Twenty Nineteen Theme and no plugins.

In this state, my website is running the default theme without any plugins. As a result, we get excellent performance. In addition, we also improve a few other metrics as well:

  • First Contentful Paint: 1.8 s
  • Speed Index: 4.2 s
  • Time to Interactive: 2.3 s
  • First Meaningful Paint: 2.0 s
  • First CPU Idle: 2.1 s
  • Maximum Potential First Input Delay: 110 ms

In terms of opportunities, the tool mentions:

  • Improve Server Response Times (TTFB): 1.14 s
  • Removed Unused CSS: 0.91 s
  • Eliminate Render-blocking Resources: 0.22 s
  • Minify CSS: 0.17 s

Clearly, we’ll never be perfect, but I do think this gives us a nice best-case scenario. Regardless, I decided to run this again on our test page:

Control WordPress Article Performance Test
Same idea! Just on the Lisp post.

Interestingly, it seems that mid-80s are about as good as we’re going to get in terms of performance. At any rate, here are the diagnostics:

  • First Contentful Paint: 2.0 s
  • Speed Index: 4.6 s
  • Time to Interactive: 4.1 s
  • First Meaningful Paint: 2.0 s
  • First CPU Idle: 2.7 s
  • Maximum Potential First Input Delay: 300 ms

Then, in terms of opportunities, we have the following:

  • Defer Offscreen Images: 2.17 s
  • Serve Images in Next-gen Formats: 1.65 s
  • Reduce Server Response Times (TTFB): 1.0 s
  • Remove Unused CSS: 0.45 s
  • Eliminate Render-blocking Resources: 0.27 s

In other words, there’s surprising still a lot I can do in terms of performance even with everything turned off. That’s good to know!

Theme

At this point, I decided to reactivate my theme to see what sort of effect it has on my website performance:

Base WordPress Homepage Performance with Theme
In this execution, all I had loaded was the theme—no plugins.

Clearly, the theme took its toll. In particular, I dropped about 20 points in performance just from activating it, so this is my knew ceiling—assuming I keep the theme. Of course, let’s take a look at the diagnostics:

  • First Contentful Paint: 2.3 s
  • Speed Index: 5.1 s
  • Time to Interactive: 4.8 s
  • First Meaningful Paint: 3.6 s
  • First CPU Idle: 4.2 s
  • Maximum Potential First Input Delay: 200 ms

Then, in terms of opportunities, it looks like most of our issues are image related (so perhaps that’s on me):

  • Serve Images in Next-gen Formats: 7.05 s
  • Efficiently Encode Images: 3.6 s
  • Eliminate Render-Blocking Resources: 1.16 s
  • Remove Unused CSS: 0.6 s
  • Defer Offscreen Images: 0.15 s

Of course, I’m more interested in seeing the results on an actual article, so I launched up that Lisp article once again:

Base WordPress Article Performance with Theme
Again, theme and no plugins—this time on an article.

Interestingly enough, we actually get a speed improvement. Of course, I believe this is related to the lack of image issues. See for yourself:

  • First Contentful Paint: 2.4 s
  • Speed Index: 4.0 s
  • Time to Interactive: 3.9 s
  • First Meaningful Paint: 3.9 s
  • First CPU Idle: 3.9 s
  • Maximum Potential First Input Delay: 40 ms

Oddly enough, I actually felt like this page loaded noticeably quicker than normal, so that’s a bonus. At any rate, here were the tips:

  • Eliminate Render-Blocking Resources: 1.23 s
  • Remove Unused CSS: 0.9 s
  • Reduce Server Response Times (TTFB): 0.55 s
  • Serve Images in Next-gen Formats: 0.45 s

Overall, I think this page gives a better idea of where I might be making some mistakes on my website. Sure, this theme has some bloat, but I think the plugins seem to be the bigger issue here.

Plugins

At this point, it’s probably a good idea to start looking at which plugins are making the most impact. Of course, I won’t bore you with the details, so I’ve distilled my testing down to the following table:

TestPerformanceFirst Contentful Paint (s)Speed Index (s)Time to Interactive (s)First Meaningful Paint (s)First CPU Idle (s)Max Potential First Input Delay (ms)
Ad Inserter752.73.85.63.85.4100
Akismet Anti-Spam902.42.83.63.63.640
Akismet Privacy Policies902.42.83.63.63.640
Brave Payments Verification892.43.33.53.53.540
Breadcrumb NavXT902.42.83.63.63.640
Broken Link Checker902.42.93.63.63.640
Easy Table of Contents812.43.25.13.65.060
Enlighter782.63.65.53.64.9120
Featured Image in RSS for Mailchimp902.43.03.53.53.540
Jetpack542.74.712.44.34.3170
List Categories892.43.03.63.63.640
List Category Posts822.42.85.04.44.780
Media File Renamer892.43.13.63.63.640
No Plugins912.42.83.53.53.540
OnePress Plus872.43.04.04.04.050
Organize Series902.42.93.63.63.640
Patreon WordPress892.43.13.63.63.640
Remove Admin Toolbar902.42.93.53.53.540
Share Buttons by AddThis742.43.67.23.63.6130
TablePress902.42.93.63.63.640
VaultPress892.43.23.63.63.640
W3 Total Cache962.12.23.02.82.860
Wordfence Security832.64.43.63.63.650
WP Author Slug892.53.03.63.63.640
WP Last Modified Info892.43.23.53.53.540
Yoast SEO892.43.23.63.63.650

As it turns out, there are really only a handful of plugins that were seriously impacting my site’s performance. By far, the most detrimental plugin was Jetpack. Just activating it caused my website performance to drop by nearly 40 points. Naturally, I decided to tackle that first.

Addressing the Problem

After looking at all this data, I realized that there were a few quick ways to make some fixes. First, find out how to limit Jetpack’s effects. Then, start trimming some of the plugin fat.

Disabling Jetpack Modules

Primarily, I looked at Jetpack and how I can limit its impact on my site. After turning off a few features, I was able to increase my homepage performance metric by nearly 20 points:

New homepage performance with several Jetpack features disabled.

Of course, who cares about the homepage? How about an article?

New article performance with several Jetpack features disabled.

Apparently, the performance is slightly better. Although, I’d argue that it’s negligible.

Removing Slow Plugins

As a result, I decided to do a bit of housecleaning. For instance, I removed the sharing buttons:

Article performance after removing the AddThis Sharing Buttons plugin.

At that point, I decided to remove my Ad Inserter plugin which gave me a noticeable improvement in performance:

Article performance after removing the Ad Insert plugin.

Taking Another Swipe at Jetpack

Then, I decided to remove even more features from Jetpack including Infinite Scroll (which I wasn’t using) and Comment Likes (which are unnecessary):

Article performance without Jetpack’s Infinite Scroll and Comment Likes features.

Again, there was only a slight boost in performance, but I’ll take it! At that point, I removed the JSON API as well as Site Stats (sadly) from Jetpack:

Article performance without Jetpack’s JSON API and Site Stats features.

Clearly, we’re seeing some improvement, so I decided to push forward.

Removing Even More Plugins

Next, I removed the List Categories plugin which didn’t really have much of an impact, but Gutenberg has the feature built in:

Article performance without List Categories plugin.

Again, we see a very minor improvement, but all these changes are trending in the right direction. After that, I removed the Health Check plugin that we used to collect our data:

Article performance without Health Check plugin.

Clearly, the removal of that plugin had little to no effect, but it felt good as a minimalist. After that, I decided to remove the List Category Posts plugin which I really was only using in three places. As a result, I was able to convert those areas to static text while happily removing the shortcodes as well as the plugin:

Article performance without List Category Posts plugin.

Again, we’re not seeing an obvious improvement in performance, but I think the performance is quite a bit more stable.

Optimizing Database and Images

At this point, I decided to run the WP-Optimize plugin to clean up the database a bit. For context, this plugin removed 649 post revisions, 637 transient options, 23 posts from trash, 46 unused comment meta data by Akismet:

Article performance after database optimization.

Clearly, this plugin didn’t add much value. That said, the plugin also offers an image compression service, which I decided to take advantage of:

Live execution of image compression using WP-Optimize.

Unfortunately, this process takes forever—over two hours for me! And, it doesn’t look like it did too much. In total, I only saved about 35 MB of data, and the compression ratio was less than 15%. That said, I felt like I might as well try it and see if it improves my website performance:

Article performance after image compression using WP-Optimize.

Again, we’re not really seeing much of an improvement anymore.

Giving Up on Advertisements

Then, I had a sudden realization: I have Jetpack ads (WordAds) and Google Adsense turned on. After disabling those, I was able to boost my site speed by a mile:

Article performance after removing advertisements.

Now, that’s awesome! These were the kind of metrics I was getting with a completely stripped down site, so I know I can’t really get any better without changing the theme or tweaking the cache.

And with that, I decided to stop! No amount of ad revenue from those two networks was going to make the performance hit worth it.

Lessons Learned

After taking a couple days to really address my website’s performance, I learned a few things about how to tune up a WordPress website.

First, I bet your WordPress website is slow. When you work with a website often, it becomes cached. As a result, it can run quickly for you. However, your users aren’t always that fortunate. In other words, their experience could be much worse, and you might not even know it. That’s why it’s important to pay attention to the following lessons.

Second, it’s a good idea to run a control audit on a page in incognito mode. If done properly, you’ll be able to see just how slow your site is without having to deal with the effects of browser plugins. In addition, this gives you a performance floor. You can only improve from there!

Third, take a good look at your plugins. Do you really need all of them? Honestly, I’ve added and removed a lot of plugins on my site over the years, but I’ve never tried to actually isolate their performance impact. Once I did, I realized that only a couple of my plugins were really causing me trouble.

Finally, consider removing ads from your site. As someone who has been blogging for nearly 3 years, I was tired of not getting compensated for my work. Of course, no amount of ad revenue is worth the performance hit they incur. Unless you’re getting enough traffic to jump on a premium ad network, I wouldn’t even bother with it. Stick to affiliate marketing or creating info products.

With these three rules in mind, I was able to dramatically improve my website performance. Sure, it’s not perfect, but I think it’ll pay dividends moving forward.

Future Plans

With my website back up to speed, my focus now is probably going to be on YouTube. That way, I can generate revenue using advertisements without any negative impacts on my site. In addition, I plan to build up a store again—something I failed at awhile back.

If you found this article helpful, let me know in the comments! Then, head on over to Patreon and pick a tier that’s right for you. Otherwise, hop on the mailing list, so we can stay in touch. In the meantime, check out some of these related articles:

While you’re hear, check out some these awesome products:

And with that, I thank you for taking time out of your day to check this site out. I appreciate it. Take care!

The post My WordPress Website Is Really, Really Slow appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/meta/my-wordpress-website-is-really-really-slow/feed/ 0 18754
AT&T Is a Disaster https://therenegadecoder.com/blog/att-is-a-disaster/ https://therenegadecoder.com/blog/att-is-a-disaster/#respond Fri, 11 Oct 2019 14:00:07 +0000 https://therenegadecoder.com/?p=18657

As if things couldn't get worse, AT&T is continually screwing me. In this episode, AT&T decided not to even show up as scheduled.

The post AT&T Is a Disaster appeared first on The Renegade Coder.

]]>

Awhile back, I wrote a trilogy complaining about AT&T, and I figured that my frustrations with them had met some sort of equilibrium. In other words, it really couldn’t get worse, could it?

Table of Contents

AT&T Refresher

In case you haven’t had the chance to read the following articles, I figured I’d recap my experience with AT&T:

First, I picked up AT&T when I moved to Atlanta in the summer of 2017. At the time, I was pretty happy with the internet because it was cheap and better than I anything I had before.

Eventually, however, I was forced to move for school. At that point, I ended up in Columbus at an apartment complex which only offered AT&T. For me, that was fine since I already had AT&T. As a result, I just had to bring the modem with me.

Unfortunately, that’s where the fun ended. First, I was forced to drop my plan to a lower tier for the same price. Then, I was fined for not returning my modem even though I was using it.

When I decided to reach out to some of their competitors, I found out that AT&T had a monopoly on my complex through a contract. In other words, every single unit had to have AT&T—literally no competition.

Of course, I eventually did get a better internet tier, but I still was pretty irritated with AT&T. Unfortunately, they didn’t really do anything to help out.

Latest Fiasco

A couple months ago, I was kicking off my second year of my PhD program by attending a pretty large event on campus called Buck-i-Frenzy. At the time, I was wandering around the event trying to pick up free swag like hockey tickets and pop sockets.

AT&T Employee Behavior

As I was rounding the last bend, I noticed that AT&T had a booth. For whatever reason, I was feeling a little frisky, so I decided to go to their booth to start some shit.

While I waited in line, I realized that their booth was the only one on campus that wasn’t just giving stuff away. In order to get a chance to spin their stupid prize wheel, you had to open an account with them. That’s a bit ridiculous.

Apparently, a lot of students were annoyed after waiting in line only to be confronted by a salesman. As a result, a lot of students were pretty candid with phrases like “can I just spin the wheel?” and “I live in the dorms.”

Of course, this irritated the AT&T employees, so one of them came up with a “genius” idea: you can only spin the wheel if you open an account or scream “Go Blue!”. Naturally, no one took them up on that offer, so they lost most of their potential customers.

My Turn

By the time I got to the front of the line, I was also pretty candid with my opening line: “do you know if fiber is ever going to come to my area?” Naturally, they couldn’t answer that question without looking up my account. At that point, they told me all the same information I already knew:

  • I have the 50MB package with a 1TB data limit
  • All units in my complex have either the 25MB package or the 50MB package
  • No fiber in the plans

However, they also told me a few things that I didn’t know:

  • My modem wasn’t good enough for the speed I was getting
  • Data limits start to throttle bandwidth at 70% usage
  • There’s a TV bundle for $10 extra which removes data limits (and I could get my modem replaced)

Obviously, I caved to the last statement and agreed to get the new package. After all, I upload data all the time. It would be nice not to have to worry about data limits.

At that point, they set me up with an appointment on September 6th (about two weeks from the time I met with them) from 9 – 11 AM. All I had to do was wait.

AT&T Visit?

About two weeks later, I got up earlier than usual in anticipation of a call from AT&T. After all, their technicians are supposed to call before they show up. Since I was the first stop of the day, I figured they would actually show up on time.

By 9:30 AM, I was starting to grow a little worried, so I called the employee I had spoken to previously. Obviously, they didn’t pick up because why would anyone at AT&T do their damn job?

After leaving a calm voicemail, I was hoping to hear back. At the time of writing, it’s been a week, and I haven’t heard anything. Not only did the technician never call or show up, but the person I called didn’t even call back.

Now, I’m sitting here wondering if I should even bother trying again. After all, they already wasted a half day. Do I let them waste more of my time, or do I stick it out with their trash service until I can move?

How Are They So Bad At Their Job?

You would think that a company that is as massive as AT&T would have their shit together, but they don’t. In fact, I think their business model relies heavily on screwing over their customers. That way, they can offer them some “solution” that costs even more money.

If I had literally any other choice over AT&T, I would take it in a heartbeat. Hell, I’d take Spectrum at this point, but I can’t even do that. At this point, it’s almost worth moving just to get some kind of internet service that isn’t complete garbage.

At any rate, thanks for taking some time to enjoy this rant. Hopefully by next year, I’ll be living somewhere else with respectable internet service. Until then, expect me to blast AT&T on twitter on a regular basis—just like John Oliver:

If you enjoyed this rant piece, check literally any of my other rant pieces. They’re likely far angrier than this one:

As always, thanks for sticking around, and don’t forget to hop on the mailing list or become a patron!

The post AT&T Is a Disaster appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/blog/att-is-a-disaster/feed/ 0 18657
I Failed My Qualifying Exam https://therenegadecoder.com/blog/i-failed-my-qualifying-exam/ https://therenegadecoder.com/blog/i-failed-my-qualifying-exam/#respond Mon, 07 Oct 2019 14:00:34 +0000 https://therenegadecoder.com/?p=18660

At just a year into my program, I've managed to hit my first bump in the road: I failed my qualifying exam. Now what?

The post I Failed My Qualifying Exam appeared first on The Renegade Coder.

]]>

It’s official! I failed my qualifying exam, and naturally I want to talk about what that means.

Table of Contents

Background

If you’ve been following along, you know all this already, but basically my PhD trajectory looks like this:

  1. Take ~30 credits
  2. Pass qualifying exam
  3. Make dissertation plan
  4. Pass candidacy exam
  5. Publish stuff
  6. Defend dissertation
  7. Graduate

At this point, I’ve just barely scratched the surface on my PhD journey. In fact, I haven’t even passed qualifying, so I have a long way to go.

Exam Content

Of course, for those of you that don’t know, the qualifying exam is basically three exams that cover topics from three courses I have already taken. In particular, the exam is on Algorithms, Programming Languages, and Operating Systems.

In an effort to study, I put together a series of articles that covered some the topics that I expected to see including:

As you can see, I did a terrible job of predicting topics for the exam. Instead, topics that were actually covered included:

  • Maximum Flow & Minimum Cut (specifically how to construct a minimum cut from a maximum flow)
  • Virtualization (specifically host physical memory, guest physical memory, and guest virtual memory and how to limit host physical memory usage)
  • Deadlocks (specifically determine if criteria were enough to protect against deadlocks)
  • Axiomatic Semantics (specifically coming up with weakest precondition and applying axioms to prove it)
  • Operational Semantics (specifically generating a new rule for do while loops)
  • Attribute Grammars (specifically writing an attribute grammar to handle features of imperative languages using a symbol table)
  • Dynamic programming (specifically something related to longest common subsequence but was maximum sum increasing subsequence)
  • Data Races in Distributed Systems (I don’t remember the problem)
  • Database Transactions (specifically determining if logs are serializable using a serialization graph)

Naturally, I’ll need to cover all these gaps. As result, get ready to read many, many more articles related to my studies.

So Close, Yet So Far

After finding out that I had failed, I ran into my advisor who wanted to chat with me about it. Almost immediately, they mentioned that they felt that I should have passed which was a relief. After all, my exam breakdown looked something like the following:

  • Algorithms: ~40
  • Programming Languages: ~75
  • Operating Systems: ~65

Here, I used approximations because I don’t know exactly what my scores were, and I can’t remember what my advisor said. At any rate, the averages were as follows:

  • Algorithms: ?
  • Programming Languages: ~55
  • Operating Systems: ~65

In other words, I absolutely killed that programming languages exam and hit the average for the operating systems exam. As for the algorithms exam, I obviously did poorly, but I’m not sure just how bad it was. Unfortunately, that was the exam that caused me to fail.

As I’ll explain below, it looks like this exam isn’t going away for me either. Moving forward, I’ll need to focus on algorithms big time.

Reflection

Honestly, I wasn’t really all that surprised that I failed. After all, I had already taken these courses, so I had a feel for how I stood in the department. In particular, I regularly failed the algorithms exams, so I wasn’t really expecting anything to be different.

That said, I did read the textbook which is an absolute first for me. Of course, the professor who wrote this exam has a few favorite topics that aren’t always covered in the book. In fact, several of their problems are related directly to topics only covered in their lectures, so I’ll need to pay attention to the lecture material in the future.

In addition, I think I’m going to have to brush up on my fear of math. Sadly, I haven’t taken a math class since 2014, and I haven’t really needed any of that since—except for this algorithms exam, of course. Maybe if I were more comfortable with math, I’d be doing alright in these theory courses.

Finally, my main takeaway from this is that everything is going to be okay. It’s just an exam. I can’t let it dictate my life.

Plan Moving Forward

As soon as I found out that I had failed the exam, I immediately started thinking about my exit strategy. After all, my plan was to jump departments if I had failed.

Then, something weird happened. My advisor, who had previously supported me in my transition, suddenly wanted me to stick it out and take the exam again. On top of that, they encouraged me to work with another Masters student to put together some research. That way, I’d be in a better position to pass the second time around.

At that point, I thought “well, now what?” Obviously, I don’t want to stick it out, but it seems like that’s going to be the best move for me. As a result, I’m going to need to juggle research, this exam, and the transition at the same time.

Luckily, I got a chance to chat with a few guys who had just made the transition, and they made it seem not so bad. Likewise, I still have funding through the end of this year, so it doesn’t make sense to take the risk right now.

At the moment, the transition timeline looks like next Fall. In the meantime, I’m going to keep grinding, so hopefully I pass the exam in the spring. If not, it’s no big deal. I’ll just have to petition for a Masters.

After that, it’s just a matter of doing some paperwork. For now, I think the best option for me will be to dual enroll. That way, if I don’t get in, I can continue my program in CSE.

Back to Work

With the exam out of the way, I feel oddly relaxed. Of course, I’m sure I’ll be swamped with work shortly, but it’s been nice not to have something looming overhead.

At any rate, hopefully I have better luck next time. For now, I’m just going to keep building up that support network! In the meantime, why don’t you show me some support by becoming a patron or hopping on the mailing list? Likewise, you can always stick around and check out these related articles:

With all that said, thanks for sticking around. I appreciate it!

The post I Failed My Qualifying Exam appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/blog/i-failed-my-qualifying-exam/feed/ 0 18660
How to Nuke Your RSS Feed https://therenegadecoder.com/blog/how-to-nuke-your-rss-feed/ https://therenegadecoder.com/blog/how-to-nuke-your-rss-feed/#respond Fri, 04 Oct 2019 14:00:22 +0000 https://therenegadecoder.com/?p=18410

Recently, I went a couple weeks without any updates due to a bug in my RSS feed. With this one simple trick, you too can nuke your RSS feed.

The post How to Nuke Your RSS Feed appeared first on The Renegade Coder.

]]>

If you’ve been subscribed to my newsletter for a little while, you may have noticed a few weeks where I wasn’t sending an emails. As it turns out, I managed nuke my RSS feed, and you can to with this one simple trick.

Table of Contents

Backstory

In case you’re not a regular subscriber, here’s the situation. Every Friday at noon, I send an automated email to all of my subscribers using MailChimp. In case you’re wondering, that email is generated using my RSS feed. Specifically, each week an email is created from the two posts from the previous week.

Well, at some point, my regular weekly email went dark:

As you can see, I sent an email out on August 9th, then I didn’t send another one out until I corrected the issue on August 24th. In other words, there were two weeks of missed emails.

At one point, I was at out of the house when I got an email saying that I had a new subscriber. Suddenly, I realized that I hadn’t gotten one of my own emails in awhile, so I decided to investigate. Yup, my automated email has been dead for some time.

Investigation

When I went home, I decided to hop on my MailChimp account to see if I could find any issues. Since I didn’t have any notifications, I thought it might be a good idea to do what IT usually does: turn my automated email off and on. Unfortunately, I couldn’t turn the email back on.

According to MailChimp, I didn’t have a proper RSS feed. In the past, one of my plugins messed up my feed, so I was thinking maybe that was the issue. Of course, I didn’t trust MailChimp, so I grabbed my RSS feed and dropped it into an RSS validator.

When I did that, I found that MailChimp was actually telling the truth. Apparently, there was some part of my feed which wasn’t properly formed. Fortunately, the validator was able to give me the exact line. When I investigated further, I realized it was an issue with a singular character in one of my articles.

If you follow that link, you’ll be taken to a pretty innocuous article. In general, the premise of the article is to share some details about the different primitive types in Java. Of course, with each primitive type, I decided to share their range. That’s when I ran into a problem.

One Tip to Nuke Your RSS Feed

If you’re familiar with Java, then you’re probably aware that characters aren’t 8-bit. Instead, they’re 16-bit which means their range extends from 0 to 65,535. Of course, not all of those characters are valid.

For example, in my article on the primitive types, I decided to demonstrate the range of characters with the following command:

Character.MIN_VALUE  // Prints the character at index 0
Character.MAX_VALUE  // Prints the character at index 65,535

Of course, when I wrote the article, I actually showed the character:

When I pasted that box character into an article, WordPress never really complained. In fact, it rendered exactly like that in the article, and I didn’t really notice any issues.

Unfortunately, two weeks went by before I realized the effects on my RSS feed. After I patched the issue, I turned around and pushed out an email that night.

Moral of the Story

As it turns out, there are certain characters that are deemed invalid in RSS. Unfortunately, I wasn’t able to find any documentation on this issue beyond a few low quality posts on forums.

If I had to guess, there is a list of illegal characters in the XML specification. As a result, the RSS specification inherits those illegal characters and may even add to it. If anyone knows more about this sort of issue, let me know.

At any rate, after this happened, I told one of my friends—Robert—about it, and he thought it was hilarious. Ever since, he’s been dumping that box character into all sorts of interfaces including Discord and iMessage. So far, we haven’t seen any issues. However, it does render differently depending on the platform.

After chatting, we remembered there used to be a funny character related bug in chrome which would crash the browser. In fact, a lot of people were making funny games where you would have to avoid touching buttons that would load the character. If you’re interested in that sort of thing, I found an article from 2015 that details the whole saga.

Help Grow The Renegade Coder

Well, that’s enough fun for one day. If you like this sort of content, I don’t mind sharing information about bugs I run into. Hell, that might actually be a fun new series.

Regardless, thanks again for your support. If you’d like to show a little extra, head on over to Patreon where you can toss me a couple bucks for my efforts. Also, you’re welcome to join the mailing list where you’ll receive one email a week. While you’re here, it’s probably a good time to promote some security textbooks through these Amazon affiliate links:

While you’re here, I have a ton more content where this came from:

As always, thanks for stopping by!

The post How to Nuke Your RSS Feed appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/blog/how-to-nuke-your-rss-feed/feed/ 0 18410
Making Drastic Changes in My PhD Journey https://therenegadecoder.com/blog/making-drastic-changes-in-my-phd-journey/ https://therenegadecoder.com/blog/making-drastic-changes-in-my-phd-journey/#respond Mon, 30 Sep 2019 14:00:36 +0000 https://therenegadecoder.com/?p=18520

As I continue on my PhD journey, I've realized that sometimes you have to make some changes to ensure you get where you want to go.

The post Making Drastic Changes in My PhD Journey appeared first on The Renegade Coder.

]]>

At this point in my PhD journey, I’m looking to make some drastic changes. Of course, that’s not always easy, so I’m writing an article to map out all the details.

Table of Contents

Background

At this point in my PhD journey, I am a year into the program. In terms of courses, I’ve completed all of the following:

  • CSE 5539: Computational Audition (2 credits)
  • CSE 5542: Real-time Rendering (3 credits)
  • CSE 5544: Data Visualization (3 credits)
  • CSE 6331: Algorithms (3 credits)
  • CSE 6341: Programming Languages (3 credits)
  • CSE 6431: Advanced Operating Systems (3 credits)
  • CSE 6559: Advanced Computer Graphics (2 credits)
  • CSE 8998: PhD Research CSE (4 credits)
  • MUSIC 7204.04: University Band (2 credits)

At this point, I’ve completed 17 graded credits in the CSE department for a 3.621 GPA. Then, this semester, I’ve added two more graded CSE credits with the following courses:

  • CSE 5521: Introduction to AI (2 credits)
  • CSE 6559: Advanced Computer Graphics (3 credits)
  • ENGREDU 7189.01: GTA Practicum (2 credits)
  • ENGREDU 7900: Professional Development (3 credits)

By the end of this semester, I should have 19 graded CSE credits.

Significance

At this point, you’re probably wondering why I’m laying out my course completion information. After all, what does my past have to do with my path going forward?

Well, as it turns out, my future is sort of up in the air right now. In particular, I’ve started to feel a bit anxious about doing research. After all, I’ve been going to seminar meetings every week for the past year, and I just can’t seem to get into it.

Of course, this realization sort of puts me in a tough place because my advisor is literally the person that allowed me to come the university in the first place. If it weren’t for him, I wouldn’t even be here, and that feeling made me want to make things work.

That said, I decided to explore other options recently which has opened quite a few doors for me. If you’re interested in that story, I wrote a whole other article about how my life is a simulation, so I won’t get into that here. However, I do want to dedicate the remainder of this article exploring those avenues.

Variables

At the moment, I’m sort of in a “waiting game” phase similar to when I was stuck waiting for applications to be accepted. Of course, now I’m waiting on my qualifying exam score. In addition, I have a lot of variables to consider moving forward.

Qualifying Exam

One of the biggest variables in my planning is the qualifying exam. At this point, I’ve already taken the exam, so it currently sits in a superposition of pass/fail. In other words, I don’t know how I did on the exam, and the result could be devastating.

If I fail the exam, I have to waste the next several months studying for it again, so I can hopefully pass it in March. If I’m studying, I’m not going to be doing research or anything else productive. In other words, I will literally be wasting time.

If I pass the exam, there are really no issues. All of my options are open. Obviously, this would be ideal. Of course, as you’ll see later, part of me hopes I fail it. After all, it dramatically reduces the decision making I would have to do.

Degree Program

Currently, I’m pursuing a PhD through the CSE department. However, depending on how my qualifying exam goes, it might make more sense for me to drop into a Masters program. That way, I don’t have to bother with studying for an exam. Instead, I can focus on getting at least one degree. Then, I can try transferring to another department for a PhD.

Of course, dropping into a Masters comes with its own risks. For instance, I’d have to choose a track: coursework, project, or thesis. Unfortunately, coursework is out since that requires passing the qualifying exam. In other words, I would be limited to the project track which requires 24 graded credits or the thesis track which requires 20 graded credits.

In either case, I’d need to come up with something to work on quickly, so I could “graduate” on time. Luckily, I already have some plans in the works. For instance, I’m already trying to support another Masters student in their music research.

In addition, there’s the issue of funding. If I drop into a Masters program, I may lose my funding. If that happens, I’ll have to pay a semester out of pocket to make this change, and that could be devastating to me financially.

Department

Another variable to add to this mess is the department. Currently, I’m in the CSE department, but I’m interested in possibly transferring to the Engineering Education Department. Unfortunately, that means that I’d have to complete a transfer application which requires letters of recommendation.

If I choose not to transfer, then I’m stuck praying that I pass the qualifying exam. See my dilemma? Again, passing the qualifying exam would remove quite a bit of stress from the decision making process, but that’s out of my control now.

Of course, I’m already leaning toward switching departments. After all, one of the biggest benefits of switching departments is the culture change.

Right now, I feel like the CSE department is pretty cold and competitive. In fact, in one year, I haven’t made a single friend. To make matters worse, I don’t think I could give you ten names of peers. Even after going to the same seminar for the past year, I’d be hard pressed to hit that kind of number.

Yet, I’ve been in maybe 6 hours worth of engineering education classes so far, and I already know 10+ names. That kind of connection is incredibly important to me, and I’m just not getting that in my current department. Frankly, I didn’t really get any sense of community or belonging in industry either, and I think this kind of change would be good for me.

Advisors

To add some complexity to my issues, it’s possible that there’s an advisor in my current department who would let me do CS education research. If that’s true, I’d still have to pass the qualifying exam, but I wouldn’t have to complete a transfer application.

In addition, switching departments means I’d have to pick up a new advisor. Naturally, that means I need to take the time now to meet faculty in the other department.

Of course, the last option is to stay in data visualization where I already of a decent connection with my current advisor. Unfortunately, I’m just not sure that’s where my heart lies currently.

Paths

At this point, there are a lot of moving parts, and I have quite a bit of time to figure things out. That said, there are a few obvious paths:

Option 1 (fail and switch departments): I could fail the exam. At that point, I’d make it an effort to drop into a project-base Masters program. Then, I would spend the following semester checking off all the boxes as I begin the transfer process to EED. By fall of 2020, I’d be where my heart lies (hopefully).

Option 2 (pass and switch advisors): I could pass the exam. At that point, I could pretty much do whatever I wanted, but I would definitely explore staying in the department. That way, I don’t have to worry about not being accepted by the EED department. Instead, I could do all the same research I would be doing over there within the department. Of course, there is a pitfall in this path: the faculty I want to work with may not be allowed to advise me.

Option 3 (fail and switch advisors): I could fail the exam but decide to stick to the department anyway. With this decision, I’d still have to pass the exam, but I wouldn’t have to worry about the transfer process. In other words, I’d waste a ton of time studying, but I wouldn’t have to write essays or ask for any letters.

Option 4 (pass and switch advisors): I could pass the exam but decide to switch departments anyway. Obviously, this choice may be made for me as there may not be anyone in the department to work with. At this point, I’d probably opt for a coursework Masters, so I wouldn’t be walking away with nothing.

At this point, I’m sort of just waiting to fail my qualifying exam, but I’m really leaning into any option which allows me to switch departments. Honestly, I think a change of scenery and material will be really great for me.

Thinking Out Loud

Sometimes, I think it’s important to just map out your thoughts on paper. In this case, I had a lot of thoughts running through my head this past week, and I just wanted to write them down. Who knows where I’ll be a year from now!

As always, thanks for taking the time to follow me on this journey. If you’d like to support the cause, head on over to Patreon and jump on one of my patron tiers. If you can’t share any cash, I’d love it if you got on the email list, so we can stay in touch.

In an article like this, I don’t feel like it’s appropriate to plug any Amazon products. At any rate, see you next time!

The post Making Drastic Changes in My PhD Journey appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/blog/making-drastic-changes-in-my-phd-journey/feed/ 0 18520
My Life Is a Simulation https://therenegadecoder.com/blog/my-life-is-a-simulation/ https://therenegadecoder.com/blog/my-life-is-a-simulation/#respond Fri, 27 Sep 2019 14:00:14 +0000 https://therenegadecoder.com/?p=18423

At times, I feel like events in my life are so coincidental that I can only chock it up to a simulation.

The post My Life Is a Simulation appeared first on The Renegade Coder.

]]>

After tackling my qualifying exam, I had a question to ask myself: what was I going to do in the next stage of my PhD program. For most people, this is an easy question to answer: do research. For me, it means following the simulation.

Table of Contents

Looking for Warmth

As you can probably tell, tech is burning me out again. In fact, I just wrote an entire article about my obsession with escapism, so I was wondering what it was about tech that was causing me to feel this way.

Early in the semester, I was sitting in an Introduction to Artificial Intelligence lecture, and I noticed how empty the room felt. Of course, there were 114 seats but only 40 students, so I managed to find a seat with no one on any side of me.

As the lecture drew on, I grew quite bored and began writing this depiction of my setting. After all, who wants to listen to a guy yell out to an empty room for 90 minutes? In fact, it was so empty that his voice just casually bounced around the walls like the inside of a church.

At that point, I began to reflect a bit on my situation. After all, this sort of environment was hardly unusual to me. In fact, I’ve always felt tech was sort of cold and cutthroat. So, what can you do? You make a change.

Entering the Simulation

This semester, I decided to take a practicum course to help me in my teaching. At the time, I joked about switching to their department which apparently caught wind. By the time the first class was over, the professor reached out to me about possibly making the transition. That was Monday.

By Tuesday, I started hunting down people who could answer questions about the transition: peers, professors, coordinators, etc. In fact, I had a small email thread going with that professor to try to answer some of my questions about potentially transferring. In addition, I had contacted the graduate coordinator who was helping me map out a plan.

On Wednesday night, I was invited to an all-day workshop on computer science education. How convenient! In just three days, I managed to go from just barely toying with an idea to being fulling immersed in the possibility. I absolutely had to attend.

By Thursday, I found myself in that workshop. That’s where I ran into one of the professors that I taught for previously. They were there with a first-year professor who just so happened to have a PhD in Engineering Education. How convenient!

After chatting with them, I learned that they had done their dissertation in emotions of people learning to code. How convenient! After all, I was feeling like the community could use a boost in emotional intelligence. I probably wouldn’t even be considering the jump to a new department if I felt more at home. After chatting with them, I had a new dilemma: should I stay in the department or not?

By Friday, I met with my current advisor to share everything I had learned that week. Without hesitation, they joked that they kind of figured I was moving in that direction. Then, they started planning out paths for me, and they even promised me their support.

By the end of the week, I was starting to wonder if my life was just a simulation. After all, nothing works at that perfectly for anyone, yet here I was watching things play nicely for me.

Moral of the Story

Before this week, I was feeling pretty down. In fact, I had just taken the qualifying exam, and I didn’t feel all that great about it. To cope, I started pouring all my extra time into my new blog, Trill Trombone, in hopes of building up some kind of income stream.

Then, for whatever reasons, the pieces started to fall into place. Suddenly, I was feeling dramatically less stressed. Sure, there’s a lot of work ahead, but at least I have a plan.

In the meantime, I’ll always keep asking for support. In particular, you can follow along on my journey by hopping on my mailing list or becoming a patron.

As always, thanks for stopping by, and thanks!

The post My Life Is a Simulation appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/blog/my-life-is-a-simulation/feed/ 0 18423
I Started a YouTube Channel https://therenegadecoder.com/meta/i-started-a-youtube-channel/ https://therenegadecoder.com/meta/i-started-a-youtube-channel/#respond Mon, 23 Sep 2019 14:00:40 +0000 https://therenegadecoder.com/?p=18442

If you follow me on social media, you've probably know this for awhile, but I started a YouTube channel. Needless to say, it's scary but a lot of fun.

The post I Started a YouTube Channel appeared first on The Renegade Coder.

]]>

If you follow me on social media, you’ve probably know this for awhile, but I started a YouTube channel. As someone who was largely afraid of the prospect of recording myself, I’ve actually found it to be a lot of fun.

Table of Contents

Fear

For awhile, the idea of making a video scared me. After all, there was a lot to be afraid of. For instance, I:

  • don’t like the sound of my own voice
  • was (still am, frankly) afraid of making a mistake that makes it through the editing process
  • don’t have the equipment or software to make good videos
  • don’t have the skills to make good videos

As always, however, I realized that you have to start somewhere. Otherwise, how would anyone be where they are?

Inspiration

Why would someone like me decide to suddenly start making videos? As it turns out, there was a bigger fear lurking.

Imagine that you’re out by a lake doing a bit of camping. While you’re there, you take special care not to get too close to the water because you can’t swim. Because of you’re lack of familiarity with water, you’re naturally scared of it.

Then one morning, you wake up to find your food is gone. In addition, due to carelessness, your car is out of gas. Suddenly, you have a much bigger worry: “how am I going to survive?” Ultimately, you decide to stick it out in the woods hoping you can buy enough time for someone can find you.

As time goes by, you grow hungry because you don’t know what’s safe to eat. Eventually, you become desperate, so you decide to return to the lake which is teeming with fish. In other words, what was once scary is now a source of life.

Moral of the story: fear is relative. In my case, I had just taken my qualifying exam, so I was worried about having to drop out. In addition to starting a niche site, I thought it would be a good idea to conquer my fear of video making. Now, I have a YouTube channel.

Oddly enough, fear is an incredible motivator.

Action

Just a day after my qualifying exam, I recorded and published my first video, and you know what: I LOVED IT. After just one video, I couldn’t wait to record another. By the time you read this article, I wouldn’t be surprised if I had 4 or more videos. In fact, today I have two:

To make these videos, I used a free software called ActivePresenter. Basically, it lets me record myself and my screen simultaneously. In addition, it even lets me edit the videos.

Since all the videos were based on articles, I didn’t really have to make any scripts. Instead, I just presented the material as-is. In total, I spent less than an hour preparing and recording my videos. Unfortunately, editing takes a lot more time.

Support

To summarize, I’m making YouTube videos now! In fact, I’ve had so much fun doing it, I wouldn’t be surprised if my writing takes a hit for a little while. Oh well.

If you want to support this journey, you can always become a member of the community through Patreon. Also, I have a mailing list. Outside of that, you can always bookmark my Amazon link.

In the meantime, take care of yourself! I know I need to do a better job of that. And as always, thanks for your support.

The post I Started a YouTube Channel appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/meta/i-started-a-youtube-channel/feed/ 0 18442
I Started a Niche Site https://therenegadecoder.com/meta/i-started-a-niche-site/ https://therenegadecoder.com/meta/i-started-a-niche-site/#respond Fri, 20 Sep 2019 14:00:54 +0000 https://therenegadecoder.com/?p=18437

If you liked The Renegade Coder, you'll love Trill Trombone—a niche site by the same idiot!

The post I Started a Niche Site appeared first on The Renegade Coder.

]]>

After writing on this site for awhile, I thought it would be fun to try to take what I’ve learned to create a niche site. In particular, I decided to launch a site for one of my hobbies: playing trombone.

Table of Contents

Inspiration

Lately, I’ve been having struggling with figuring out who I want to be. After finding out that industry wasn’t for me, I tried to moving into academia. Now that I’m a year into that, I’m realizing that I don’t really like data visualization, so I’m trying to move into education.

Over time, I’ve realized that money would really alleviate this stress. After all, I wouldn’t have to worry about making any poor choices because I’d always have the money to do whatever I wanted. Unfortunately, I haven’t been able to earn much of anything on this site.

As fate would have it, I’ve been turned on to Income School which has a ton of YouTube videos on how to launch a niche site to earn money. Obviously, I broke a lot of those rules when I made this website, so I thought it would be fun to pivot into a new space with a new blog.

Action

Now, you can’t exactly have big dreams without acting on them, so I decided to pull the trigger on a new domain: trilltrombone.com. As a part of my hosting package, I was able to get this domain for free, and I even get to keep the contents in the same place as The Renegade Coder which is great.

Also, since I’m already used to working with WordPress, I decided to stick with that. However, I really made sure to pick a light (and free) theme this time around. Now, my new blog is blazing fast which should help me out in Google rankings.

In addition, I made sure to pick a topic that I could actually monetize. Unfortunately, teaching people to code isn’t all that lucrative—at least in terms of Amazon revenue. I don’t exactly write posts about products since almost everything I do is free.

With a site all about trombones, I have a lot I can monetize: trombones, slide lubricants, songbooks, mouthpieces, etc. And, I can write really focused articles about things I’m interested in.

At any rate, the site is up! By the time you’re reading this, I would expect there to be at least 10 articles, so that’s exciting. In fact, I’ve already written the following articles:

As you can see, a couple of these articles are extremely targeted. Hopefully, that means I’ll get extremely targeted traffic which means increased engagement and ultimately sales. In other words, I’ll be making some income through that site shortly—knock on wood!

Support

As is often the case on this site, I like to share updates every now and again about what I’m working on related to this site. Typically, these articles are fairly short and include updates for projects like the Sample Programs repo. Of course, I felt website building was fairly related to what I do here, so I figured I’d share this story about my niche site.

As is customary at the end of my posts, I like to panhandle, so I don’t feel like my writing is just charity. In other words, become a patron, join the newsletter, write a comment, or buy some stuff from Amazon:

Of course, whatever you do, thanks for stopping by. I appreciate your time and attention.

The post I Started a Niche Site appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/meta/i-started-a-niche-site/feed/ 0 18437
Data Science, the Good, the Bad, and the… Future https://therenegadecoder.com/code/data-science-the-good-the-bad-and-the-future/ https://therenegadecoder.com/code/data-science-the-good-the-bad-and-the-future/#respond Mon, 16 Sep 2019 14:00:45 +0000 https://therenegadecoder.com/?p=18413

For the first time in awhile, we have a bit of a community post. Check out this article on data science by Kirit Thadaka.

The post Data Science, the Good, the Bad, and the… Future appeared first on The Renegade Coder.

]]>

Hey! Jeremy here. Recently, someone from the editorial team for Kite, an AI autocomplete for Python, reached out to see if I would share some of their content. Since I thought the tool looked awesome, I figured I’d help them out. After some chatting, we decided on this data science article by Kirit Thadaka. Enjoy!

Table of Contents

What is Data Science?

How often do you think you’re touched by data science in some form or another? Finding your way to this article likely involved a whole bunch of data science (whooaa). To simplify things a bit, I’ll explain what data science means to me.

“Data Science is the art of applying scientific methods of analysis to any kind of data so that we can unlock important information.”

That’s a mouthful. If we unpack that, all data science really means is to answer questions by using math and science to go through data that’s too much for our brains to process.

Data Science covers…

  • Machine learning
  • Data visualization
  • Predictive analysis
  • Voice assistants

… and all the buzzwords we hear today, like artificial intelligence, deep learning, etc.

To finish my thought on data science being used to find this article, I’ll ask you to think of the steps you used to get here. For the sake of this explanation, let’s assume that most of you were online looking at pictures of kittens and puppies when you suddenly came across a fancy word related to data science and wanted to know what it was all about. You turned to Google hoping to find the meaning of it all, and you typed “What is *fill in your data science related buzzword*.”

You would have noticed that Google was kind enough to offer suggestions to refine your search terms – that’s predictive text generation. Once the search results came up, you would have noticed a box on the right that summarizes your search results – that’s Google’s knowledge graph. Using insights from SEO (Search Engine Optimization) I’m able to make sure my article reaches you easily, which is a good data science use case in and of itself. All of these are tiny ways that data science is involved in the things we do every day.

To be clear, going forward I’m going to use data science as an umbrella term that covers artificial intelligence, deep learning and anything else you might hear that’s relevant to data and science.

Positives: Astrophysics, Biology, and Sports

Data science made a huge positive impact on the way technology influences our lives. Some of these impacts have been nice and some have been otherwise. *looks at Facebook* But, technology can’t inherently be good or bad, technology is… technology. It’s the way we use it that has good or bad outcomes.

We recently had a breakthrough in astrophysics with the first ever picture of a black hole. This helps physicists confirm more than a century of purely theoretical work around black holes and the theory of relativity.

To capture this image, scientists used a telescope as big as the earth (Event Horizon Telescope or EHT) by combining data from an array of eight ground-based radio telescopes and making sense of it all to construct an image. Analyzing data and then visualizing that data – sounds like some data science right here.

A cool side note on this point: a standard Python library of functions for EHT Imaging was developed by Andrew Chael from Harvard to simulate and manipulate VLBI (Very-long-baseline interferometry) data helping the process of creating the black hole image.

Olivier Elemento at Cornell uses Big Data Analytics to help identify mutations in genomes that result in tumor cells spreading so that they can be killed earlier – this is a huge positive impact data science has on human life. You can read more about his incredible research here.

Python is used by researchers in his lab while testing statistical and machine learning models. Keras, NumPy, Scipy, and Scikit-learn are some top notch Python libraries for this.

If you’re a fan of the English Premier League, you’ll appreciate the example of Leicester City winning the title in the 2015-2016 season.

At the start of the season, bookmakers had the likelihood Leicester City winning the EPL at 10 times less than the odds of finding the Loch Ness monster. For a more detailed attempt at describing the significance of this story, read this.

Everyone wanted to know how Leicester was able to do this, and it turns out that data science played a big part! Thanks to their investment into analytics and technology, the club was able to measure players’ fitness levels and body condition while they were training to help prevent injuries, all while assessing best tactics to use in a game based on the players’ energy levels.

All training sessions had plans backed by real data about the players, and as a result Leicester City suffered the least amount of player injuries of all clubs that season.

Many top teams use data analytics to help with player performance, scouting talent, and understanding how to plan for certain opponents.

Here’s an example of Python being used to help with some football analysis. I certainly wish Chelsea F.C. would use some of these techniques to improve their woeful form and make my life as a fan better. You don’t need analytics to see that Kante is in the wrong position, and Jorginho shouldn’t be in that team and… Okay I’m digressing – back to the topic now!

Now that we’ve covered some of the amazing things data science has uncovered, I’m going to touch on some of the negatives as well – it’s important to critically think about technology and how it impacts us.

The amount that technology impacts our lives will undeniably increase with time, and we shouldn’t limit our understanding without being aware of the positive and negative implications it can have.

Some of the concerns I have around this ecosystem are data privacy (I’m sure we all have many examples that come to mind), biases in predictions and classifications, and the impact of personalization and advertising on society.

Negatives: Gender Bias and More

This paper published in NIPS talks about how to counter gender biases in word embeddings used frequently in data science.

For those who aren’t familiar with the term, word embeddings are a clever way of representing words so that neural networks and other computer algorithms can process them.

The data used to create Word2Vec (a model for word embeddings created by Google) has resulted in gender biases that show close relations between “men” and words like “computer scientist”, “architect”, “captain”, etc. while showing “women” to be closely related to “homemaker”, “nanny”, “nurse”, etc.

Here’s the Python code used by the researchers who published this paper. Python’s ease of use makes it a good choice for quickly going from idea to implementation.

It isn’t always easy to preempt biases like these from influencing our models. We may not even be aware that such biases exist in the data we collect.

It is imperative that an equal focus is placed on curating, verifying, cleaning, and to some extent de-biasing data.

I will concede that it isn’t always feasible to make all our datasets fair and unbiased. Lucky for us, there is some good research published that can help us understand our neural networks and other algorithms to the extent that we can uncover these latent biases.

When it comes to data science, always remember –

“Garbage in, garbage out.”

The data we train our algorithms with influences the results they produce. The results they produce are often seen by us and can have a lasting influence.

We must be aware of the impact social media and content suggestions have on us. Today, we’re entering a loop where we consume content that reinforces our ideas and puts people in information silos.

Research projects that fight disinformation and help people break out of the cycle of reinforcement are critical to our future. If you were trying to come up with a solution to this fake news problem, what would we need to do?

We would first need to come up with an accurate estimate of what constitutes “fake” news. This means comparing an article with reputable news sources, tracing the origins of a story, and verifying that the article’s publisher is a credible source.

You’d need to build models that tag information that hasn’t been corroborated by other sources. To do this accurately, one would need a ton of not “fake” news to train the model on. Once the model knows how to identify if something is true (to a tolerable degree of confidence), then the model can begin to flag news that’s “fake.”

Crowd sourced truth is also a great way to tackle this problem, letting the wisdom of the crowd determine what the “truth” is.

Blockchain technology fits in well here by allowing data to flow from people all over the world and arrive at consensus on some shared truth.

Python is the fabric that allows all these technologies and concepts to come together and build creative solutions.

Python, a Data Science Toolset

I’ve talked about data science, what it means, how it helps us, and how it may have negative impacts on us.

You’ve seen through a few examples how Python is a versatile tool that can be used across different domains, in industry and academia, and even by people without a degree in Computer Science.

Python is a tool that makes solving difficult problems a little bit easier. Whether you’re a social scientist, a financial analyst, a medical researcher, a teacher or anyone that needs to make sense of data, Python is one thing you need in your tool box.

Since Python is open source, anyone can contribute to the community by adding cool functionalities to the language in the form of Python libraries.

Data visualization libraries like Matplotlib and Seaborn are great for representing data in simple to understand ways. NumPy and Pandas are the best libraries around to manipulate data. Scipy is full on scientific methods for data analysis.

Whether you want to help fight climate change, analyze your favorite sports team or just learn more about data science, artificial intelligence, or your next favorite buzzword – you’ll find the task at hand much easier if you know some basic Python.

Here are some great Python libraries to equip yourself with:

  • NumPy
  • Pandas
  • Scikit-Learn
  • Keras
  • Matplotlib

I’ll illustrate an example of how easy it is to get started with data science using Python. Here’s a simple example of how you can use Scikit-Learn for some meaningful data analysis.

Python Example with Scikit-learn

This code is available at the Kite Blog github repository.

I’ve used one of Scikit-Learn’s datasets called Iris, which is a dataset that consists of 3 different types of irises’ (Setosa, Versicolour, and Virginica) petal and sepal length, stored in a 150×4 numpy.ndarray. The rows are the samples and the columns are: Sepal Length, Sepal Width, Petal Length, and Petal Width.

I’m going to run a simple linear regression to display the correlation between petal width length. The only libraries used here are scikit-learn (for the regression and data set) and matplotlib for the plotting.

from sklearn import datasets, linear_model
import matplotlib.pyplot as plt

iris = datasets.load_iris()

# Data and features are both numpy arrays
data = iris.data
features = iris.feature_names

Now, we’ll plot a linear regression between the length and width of the petals to see how they correlate.

# Create the regression model
regression = linear_model.LinearRegression()

# Reshape the Numpy arrays so that they are columnar
x_data = data[:, 2].reshape(-1, 1)
y_data = data[:, 3].reshape(-1, 1)

# Train the regression model to fit the data from iris (comparing the petal width)
regression.fit(x_data, y_data)


# Display chart
plt.plot(x_data, regression.predict(x_data), color='black', linewidth=3)
plt.scatter(x_data, y_data)
plt.show()

Here’s a tutorial I created to learn NumPy, and here’s a notebook that shows how Keras can be used to easily create a neural network. Just this much will allow you to build some pretty cool models.

Concluding thoughts

Before I end, I’d like to share some of my own ideas of what I think the future of data science looks like.

I’m excited to see how concerns over personal data privacy shapes the evolution of data science. As a society, it’s imperative that we take these concerns seriously and have policies in place that prevent our data accumulating in the hands of commercial actors.

When I go for walks around San Francisco, I’m amazed at the number of cars I see with 500 cameras and sensors on them, all trying to capture as much information as they possibly can so that they can become self driving cars. All of this data is being collected, it’s being stored, and it’s being used. We are a part of that data.

As we come closer to a future where self driving cars become a bigger part of our life, do we want all of that data to be up in the cloud? Do we want data about the things we do inside our car available to Tesla, Cruise or Alphabet (Waymo)?

It’s definitely a good thing that these algorithms are being trained with as much data as possible. Why would we trust a car that hasn’t been trained enough? But that shouldn’t come at the cost of our privacy.

Instead of hoarding people’s personal data in “secure” cloud servers, data analysis will be done at the edge itself. This means that instead of personal data leaving the user’s device, it will remain on the device and the algorithm will run on each device.

Lots of development is happening in the field of Zero Knowledge Analytics which allows data to be analyzed without needing to see what that data is. Federated Learning allows people to contribute to the training of Neural Networks without their data to leaving their device.

The convergence of blockchain technology and data science will lead to some other exciting developments. By networking people and devices across the globe, the blockchain can provide an excellent platform for distributed computation, data sharing, and data verification. Instead of operating on information in silos, it can be shared and opened up to everyone. Golem is one example of this.

Hypernet is a project born out of Stanford to solve a big problem for scientists – how to get enough compute power to run computationally and data intensive simulations.

Instead of waiting for the only computer in the university with the bandwidth to solve the task and going through the process of getting permission to use it, Hypernet allows the user to leverage the blockchain and the large community of people with spare compute resources by pooling them together to provide the platform needed for intensive tasks.

Neural networks for a long time have felt like magic. They do a good job, but we’re not really sure why. They give us the right answer, but we can’t really tell how. We need to understand the algorithms that our future will be built on.

According to DARPA, the “third-wave” of AI will be dependent on artificial intelligence models being able to explain their decisions to us. I agree, we should not be at the mercy of the decisions made by AI.

I’m excited with what the future holds for us. Privacy, truth, fairness, and cooperation will be the pillars that the future of data science forms on.

This article originally appeared on Kite.

The post Data Science, the Good, the Bad, and the… Future appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/code/data-science-the-good-the-bad-and-the-future/feed/ 0 18413
How to Print on the Same Line in Python: Print and Write https://therenegadecoder.com/code/how-to-print-on-the-same-line-in-python/ https://therenegadecoder.com/code/how-to-print-on-the-same-line-in-python/#respond Fri, 13 Sep 2019 14:00:23 +0000 https://therenegadecoder.com/?p=18345

While Python is often convenient, sometimes it's not. For instance, it's unclear how you would print on the same line in Python. Luckily, we have the answer!

The post How to Print on the Same Line in Python: Print and Write appeared first on The Renegade Coder.

]]>

As someone who teaches a lot of beginner programming content, I occasionally stumble upon questions like “how do you print on the same line in Python?” Luckily, I have an answer to that!

In short, there are two main ways to print on the same line in Python. For Python 2, use the following print syntax: print "Williamson",. For Python 3, use the following print syntax: print("Providence", end=""). Otherwise, check out the remainder of the article for a backwards compatible solution.

Table of Contents

Problem Introduction

In many programming languages, printing on the same line is typically the default behavior. For instance, Java has two command line print functions:

System.out.print();
System.out.println();

As you can probably imagine, the default print function in Java is going to print without a newline character. In contrast, the println function is going to behave much like the print function in Python. Specifically, it’s going to print whatever string you provide to it followed by a newline character (i.e. \n).

Of course, if the print function in Python automatically prints a newline character with each call, then there’s no way to get the Java print behavior, right? Luckily, that’s not true! Otherwise, I wouldn’t have anything to write about out.

Solutions

In order to print on the same line in Python, there are a few solutions. Unfortunately, not all of the solutions work in all versions of Python, so I’ve provided three solutions: one for Python 2, another for Python 3, and a final solution which works for both.

Print on the Same Line The Old Way

When I was searching for solutions to this problem, I found a lot of material on Python 2 which is quickly phasing out (I hope). That said, I felt this solution would be helpful to anyone still rocking it.

At any rate, when you print something in Python 2, the syntax is the same as Python 3, but you leave out the parentheses:

print "Live PD"

Of course, in both cases, the default behavior is to print with a newline. As a result, we’ll need to add a clever bit of syntax—a comma:

print "Live PD",

Now, the print function should exclude the newline. However, this solution will add an extra space to the end of the string. Also, you may notice that this solution does not print immediately. If that happens, you can make a call to sys.stdout.flush().

Print on the Same Line with the Write Function

Fortunately, we can bridge the gap between Python 2 and 3 using a function out of the sys library: write. This functions works just like the print function, but there’s no implicit newline:

import sys
sys.stdout.write("Breaking Bad")

Again, since there is no newline, you may need to flush the buffer to see any results:

import sys
sys.stdout.write("Breaking Bad")
sys.stdout.flush()

In either case, this solution will get the job done in both versions of Python.

Print on the Same Line the New Way

In Python 3, print is a standard function. As a result, it has additional opportunities for parameters. In particular, there is a keyword argument called end which defaults to some newline character. You can easily change it as follows:

print("Mob Psycho 100", end="")

And, that’s it! Instead of the string ending in a newline, it will end in an empty string. Of course, this solution comes with the same caveat as the other previous two solutions: you may need to flush the buffer.

Performance

As always, I like to take a look at all the solutions from the point of view of performance. To start, I usually store each solution in a string. To avoid excessive printing during the test, I’ve chosen to write empty strings:

setup="""
import sys
"""

write_solution = """
sys.stdout.write("")
"""

print_solution = """
print("", end="")
"""

Unfortunately, I was unable to test the Python 2 solution on my system, so feel free to share your results in the comments. At any rate, I like to use the timeit library for a quick and dirty performance test:

>>> import timeit
>>> min(timeit.repeat(stmt=write_solution, setup=setup, repeat=10))
0.20978069999999605
>>> min(timeit.repeat(stmt=print_solution, setup=setup, repeat=10))
0.5292953999999952

Clearly, the print function has quite a bit of overhead. In other words, if performance matters, go the write route. Otherwise, print works great!

A Little Recap

Well, that’s it for this one. Check out the code block below for a list of all the solutions:

# Python 2 only
print "Live PD",

# Backwards compatible (also fastest)
import sys
sys.stdout.write("Breaking Bad")

# Python 3 only
print("Mob Psycho 100", end="")

As always, if you know any other ways to print on the same line in Python, let us know in the comments. In the meantime, why not grow your Python knowledge with the following articles:

If you liked this article or any of the ones I listed, consider sticking around long term by becoming a member of the community or hopping on the mailing list.

While you’re here, why not take advantage of some these Python books:

Otherwise, I appreciate the support. Thanks for stopping by!

The post How to Print on the Same Line in Python: Print and Write appeared first on The Renegade Coder.

]]>
https://therenegadecoder.com/code/how-to-print-on-the-same-line-in-python/feed/ 0 18345