Pseudo Random

The New York Time says Data Analysts Captivated by Power of R.

R is ... the name of a popular programming language used by a growing number of data analysts inside corporations and academia. It is becoming their lingua franca partly because data mining has entered a golden age, whether being used to set ad prices, find new drugs more quickly or fine-tune financial models. Companies as diverse as Google, Pfizer, Merck, Bank of America, the InterContinental Hotels Group and Shell use it.

Hmmm, "fine tune financial models". Does R stand for Recession?

More seriously, does data mining plus multi-core machines add up to an important language direction for the next few years? How well does R fare on such boxen?

More on R previously on LtU here and here.

Jim B. stared wistfully in the mirror at the wrinkles near his eyes and the few stray gray hairs that he’d accumulated over the last six months. On the way back to his desk, he stopped by his friend Mike's desk. “Point three six,” he said as he banged his head against Mike's cubicle wall. “Point three six.”

His work was about as high-stress as it gets – he was on a team building a security system that was responsible for keeping ne’er-do-wells out of data belonging to nine-figure financial companies and an array of three-letter government organizations, many of which Jim had never heard of. But if he failed and a hacker got into the data, Jim would be sure to find out exactly what the FQD was when they had him deported.

Security at the company was no joke – all employees were subjected to a drug test, three separate interviews including one where they’d be grilled on ethics as they apply to IT, not to mention the extensive seven-page document he’d had to fill out for his background investigation to be conducted. Every door had a keycard lock, all passwords (mixed-case, special characters, minimum 10 characters) expired every 30 days, and all electronic devices (iPods and cell phones) had to be approved by management before they could be brought in. This was serious security.

Mike was really Jim's only friend on the inside. Most of the salaried staff was skeptical of contractors like Jim, but Mike didn’t subscribe to their groupthink. Jim and Mike shared in the pain of looming deadlines, the middle-school-esque giggling at management gaffes during meetings, and the sadness of the vending machine running out of Dr. Pepper. Mike was always willing to lend a sympathetic ear to the dangerous levels of silliness Jim encountered on his projects.

With security stopping just shy of cavity searches at the beginning of the day, Jim was amazed when he figured out how to break into the system with ease. Their impregnable system had just been impregnated.

Emergency Meeting

Jim called up an emergency meeting. “I was able to crack the security and get root access,” Jim began. An engineer audibly gasped.

“But how?” someone cried.

“All I had to do was modify the cookie and the lock file, and…”

“That’s stupid. Why would anyone ever do that?” His boss, Paul, shot a glare at him.

“I don’t know,” Jim said, “curiosity? Ignorance? Malice?”

“We can’t guard against malice.”

“We can’t guard against malice?!” Jim’s jaw dropped. Guarding against malice was the whole point, and the entire reason that the company existed. This would’ve been excusable if it came from a PHB, but this was a very experienced engineer.

“Guarding against malice is exactly what we're here to do! If we’re not going to guard against malice, we might as well just put post-it notes on the data that say ‘Please Don’t Look.’”

A young developer scoffed. “You can’t put post-it notes on data!” Several people around the table exchanged smirks.

Jim exhaled deeply and could feel another wrinkle forming near his eye and another tuft of hair turning gray. “Look, this is what hackers do. They dig around and find all the loose threads and try to use those to exploit the system. If I’m capable of doing it, then certainly a professional hacker paid by a government or well-funded competitor can.”

“But if they wanted to get to the database, they’d just have to do ‘psql –d xxxxxx-db-name’ and they’re in. Like I said, we can’t protect against malice!”

Jim briefly wondered if he looked half as horrified as Mike did at that moment. “You… didn’t encrypt the database?!”

“…No…”

The room remained silent, and Jim’s mouth remained agape with horror. Aside from Mike and Jim, everyone just looked bored. Jim tried to size up the situation – it didn’t make sense that no one had his back. His colleagues certainly weren’t curious, but they were perhaps ignorant or malicious. Jim scanned the room with eyes imploring someone to have his back, but no one made any eye contact except Mike, who was apparently too stunned to speak up.

“Look, I have real work to do,” one complained. “Are we done here?” A few quiet “yeah, me too”s echoed throughout the room. All of the bigwigs in the room scowled, packing up their notepads and folders. As they filed out of the room, a few stink-eyes were shot Jim’s way. “What a colossal waste of time,” one said under his breath.

Eventually only Jim and Mike remained in the room. Jim was still dumbfounded. “What the hell just happened?!”

“I really don’t know, Jim. I can’t believe it either. But hey, I have to run to CompUSA. You’ve got that meeting with management now, right?”

Jim’s heart sank. He had almost forgotten that in 15 minutes, he’d have a three hour meeting with some of the bigwigs that had just turned against him in his last meeting. At this rate of wrinkle and gray hair accumulation, he'd be looking about 30 years older by the end of the day.

Can’t Wait

After the meeting, Jim was feeling numb. He hobbled over to Mike’s desk for some consoling.

“Man, I have to say, I just want to get laid off.”

“You know, most engineers would’ve stopped that sentence a word earlier,” Mike replied with a chuckle. “I got you something from CompUSA.”

Mike had picked up a beanbag keyboard rest, that he set in the spot where Jim would beat his head against Mike’s desk in moments of intense frustration.

“Point three six, man,” Jim said while banging his head against the wrist rest. “Point three six.” It was his way of reminding himself that August 22nd (or 8/22 ≈ .36) was the end of his contract.

Brought to you by the Non-WTF Job Board:

"That's nothing to smile at, Brett," writes David Robinson, "EntityName parsing errors are serious business."

(currently live at NBC Sports)

 

"That's okay," Zack notes, "I can wait."

 

"We're all familiar with the 'marketing' prices of $299.99 instead of $300," Alexandre Hetu writes, "but I'm not quite sure how the random question mark is supposed to work."

Translates to "Condos from"

 

"I would like to comply with their demands," writes Tim, "but I'm not really sure how. Maybe these guys need to rent a coder?"

 

"When setting up Windows on a new HP desktop," Andrew noted, "it seems they like to crush your high hopes with an offer you can't refuse."

 

Aaron writes, "I was playing word scramble with the AIM chatbot SmarterChild, and I got this:

SmarterChild: That's correct.
  You are 3 for 4.
Try this one: LVIDI.

Aaron: livid

SmarterChild: Right again!
  You are 4 for 5.
Here we go: QEREU.

Aaron: queer

SmarterChild: I don't like the way you're speaking now. These are awful words to use.

"Asking why it had me unscramble a bad word didn't yield any meaningful answer."

Brought to you by the Non-WTF Job Board:

Some time ago Martin F. was sent in by his IT consulting company to help fix some problems with the HR Database at a major European banking / insurance firm.  He admits that the WTF worthy warning signs were there at the onset (among them being that he was the 4th in a series of consultants assigned to this project), but being relatively naive to such things, he accepted the position and spent a year shaking his head in bewilderment and, at the same time, his fists at Rob.  He was an HR 'specialist' and a true IT genius who had a self-proclaimed hobby of programming in Visual Basic and was, of course, long gone from the corporation.

The HR Access-pool

The problem database held monthly snapshots of information about all the (over 50,000) employees of the bank, such as their names, DOBs, home address, function, fixed and variable salaries going back about 4 years.  Over that period of time, it had expanded to an impressive 2 Million records in size which is not unheard of in any large corporation, but your typical "Select and Group By" query on 200 people took about eight minutes.  However, as Martin came to discover, this was mostly due to the fact the corporate HR database was in reality an Access database sitting on some network fileshare.

Feeding-wise: every month, some 100 local providers from daughter companies would fill-in an Excel-based tool (created by the infamous Rob) that exported CSVs and send them by e-mail to the manager of the application. He would, in turn...

1. Process every file through a series of checks implemented in Ms Access. That would export new "Verified CSVs"
2. Consolidate all the verified files into another Access database
3. Find and remove people who appeared twice and do the necessary corrections (to avoid them being counted as multiple FTEs). Overwrite the "verified CSVs" with the result
4. Import the CSV's into temporary tables of the master database
5. Perform some more checks and corrections
6. Consolidate the temporary data into the master tables
7. Export all the valuable statistics to Excel files for those who needed them (mainly the financial reporting).

This song and dance took 2 people a week to accomplish. After that was done: leave 3 days for feedback and corrections from the providers, then go to step 1 and repeat the process for the "final" version. 

Ol' Teller

One of Martin's first tasks was to correct some flaws in the Excel-based input tool (which every provider had a different version of since problems were dealt by e-mail on a case-to-case basis).  Under the spreadsheet's ordinary looking thin veneer was a writhing dung heap of undocumented VBA code and WTFs galore that could possibly stand on its own as a separate CodeSOD submission, but for Martin, one procedure stood out as being truly a case of doing something completely wrong.

Private Function Row_Is_Empty(iRowNr As Long, lLastCol, sh As Worksheet) As Boolean  
   Dim j As Integer
   Dim is_empty As Boolean
   Dim teller As Integer
   teller = 0
   Dim curr_cell As String
     
   For j = 1 To lLastCol
         curr_cell = Trim(Replace(sh.Cells(iRowNr, j).value, Chr(10), ""))
      If curr_cell <> vbNullString Then
        teller = teller + 1
      Else
        teller = teller
      End If
   Next j
 
   If teller = 0 Then
   is_empty = True
   Else
   is_empty = False
   End If
 
   teller = 0
   If is_empty = true Then  
   Row_Is_Empty = is_empty
   Else
   Row_Is_Empty = false
   End If
   is_empty = False
   teller = 0
End Function
 

 

Brought to you by the Non-WTF Job Board:

The Bug That Shut Down Computers World-Wide was originally published on October 2nd, 2007, and today marks the twenty-fifth anniversary of the story. Here’s to a great 2009!

Where were you the morning of January 1st, 1984? Me? I was out living it up at Divestiture-fest ’84 and – let me tell you – it was quite a party. We drank until those seven little Baby Bells looked like fourteen, and kept drinking until it all looked like AT&T again. Ah, the good old days. But not all of us were out celebrating. Some – like Robert Reagan – were actually working, desperately trying to fix the bug that shut down computers across the world.

With all the “oh no, the world’s gonna end” date problems out there – Y2K, DST, The End of the Epoch, and Y2070 – it’s surprising that most haven’t heard of the day that the world actually ended. On that day – January 1st, 1984 – a single bug was responsible for shutting down – and keeping down – a whole lot of computer systems.

There were three big rages back in the 1980’s: Rubik’s cubes, Molly Ringwald, and Wangs. While I’m sure you are all familiar with those first two 80’s icons, Wangs – i.e., any one of the many systems from Wang Laboratories – have fallen into the void of boring, obsolete computer junk. But a quarter century ago, Wang was where it was at.

Around 1983, Wang Laboratories employed upwards of 30,000 people and brought in revenues of nearly $6 billion/year (in today’s dollars). And their computers were everywhere. The Wang OIS (Office Information System) was perfect for many businesses and allowed “easy extensibility” through its Glossary programming language. It was so simple, apparently, that secretaries were able to master it.

Wang was even able to compete with IBM’s System 34 and System 36’s with their own line of mini-computers called the Wang VS. These systems hosted the standard development tools – COBOL, BASIC, RPG, etc – and its own. And like many Operating Systems of Old, the Wang VS was a little peculiar.

One interesting security feature of the Wang VS was File Expiration Dates. In addition to having a Creation and Modification date, files included an Expiration date. If someone – regardless of his/her security privileges – attempted to delete a file not past expiration, the operating system would raise an error. Although users could easily change the expiration date with another command, it was seen as a “nice to have” that prevented accidental deletions.

Like many operating systems, the Wang VS and its applications utilized temporary files to do all sorts of things, from caching to scheduling.

Like many modern development environments, there was no built-in mechanism for date/time calculations. Different modules of the operating system and its applications relied on different functions for different date/time calculations. Not surprisingly, one of these functions had a bug.

Unfortunately, the bug was in the temporary file creation module. Specifically, it mistakenly incremented the zero-indexed day-of-year for all day-of-years that are equal to 0 in a leap-year. In other words, when January 1, 1984 (day 0 in a leap year) hit, the code set “January 2, 1984” as a temporary file’s expiration date.

Because temporary files are created and deleted in a manner of seconds, the operating system was blocked from deleting its own files. One by one, processes in the Wang VS shut down and the computers grinded to a halt. Since boot time activity included the creation and deletion of temporary files, the operating system boot process could not complete. This effectively put ever single Wang VS computer in an unbootable state.

Wang engineers and support techs worked around-the-clock to restore and fix everyone’s broken computer. Thankfully, a combination of the New Year’s Day holiday and FedEx (who, apparently, were all using IBM computers) overnight shipping, most businesses only suffered a minor disruption of service. And the day the world ended – January 1st, 1984 – passed like every other.

Brought to you by the Non-WTF Job Board:

The Harbinger of the Epoch was originally published on June 20th, 2006. Happy New Year, and have fun fixing those "Date Not Found" bugs created by your predecessors who never would have thought their software would see 2009.

January 19, 2038 is a date which will live in infamy. It is on that day that the 32-bit integer storing the number of seconds since the beginning of The Epoch will overflow, causing death and destruction unseen by the world since the Y2K Bug. As they did in 2000, software will spectacularly crash, hardware will explode, appliances will go haywire and attack their owners, and nuclear missiles will simultaneous launch and destroy the world. Casper Kvan was reminded of this impending doom when one of his systems suddenly went down.

The system that crashed was a vendor application and failed as a result of the database shutting down. Easy enough to fix, Casper restarted the database. It came back up, then went down again. He restarted it again, it came up, then went down. Up, down, up, down. The log files revealed the problem was in the vendor's startup script, which went something like this:

 

start oracle sleep 1000000000 stop oracle

It was pretty clear why the script failed. Time passed 2^31-10^9 that weekend, so the sleep command was overflowing and terminating, and thus the database was shut down. No one, including the vendor, could explain why their startup script issued 30+ year sleep command and then a shutdown, but the fix was pretty simple: the vendor shaved a zero off the sleep command.

Brought to you by the Non-WTF Job Board:

A Function to Quit For was originally published on July 17, 2006

It's easy to understand how Mike Hartnagel got himself into the classic "should I quit after three days?" dilemma. Who would have guessed that a single bullet point on the job description -- "utilizing some Excel Interoperability" -- actually described the architecture of the system: a horrid amalgamation of Excel spreadsheets interacting with C# interacting with other spreadsheets. Who would have thought that all other bullet points -- from "multi-tiered architecture using remoting" to "automated/integrated build process" -- were actually features they'd like to have at some point in the distant future.

Who? A hardened cynic -- that's who. And that's exactly what Mike became after being handed his first programming assignment: create overloads for the following function so that developers don't need use zeros if they don't need all of the 600+ parameters ...

 

/* adds up all the column values for a specific row from an excel file. * 20050823 - BSR - v1.0 - Adds only columns a to z. * 20050909 - BSR - v2.0 - Added support for adding columns aa to zz. */ public long addUpAllExcelColumns( int a, int b, int c, int d, int e, int f, int g, int h, int i, int j, int k, int l, int m, int n, int o, int p, int q, int r, int s, int t, int u, int v, int w, int x, int y, int z, int aa, int ab, int ac, int ad, int ae, int af, int ag, int ah, int ai, int aj, int ak, int al, int am, int an, int ao, int ap, int aq, int ar, int _as, int at, int au, int av, int aw, int ax, int ay, int az, int ba, int bb, int bc, int bd, int be, int bf, int bg, int bh, int bi, int bj, int bk, int bl, int bm, int bn, int bo, int bp, int bq, int br, int bs, int bt, int bu, int bv, int bw, int bx, int by, int bz, int ca, int cb, int cc, int cd, int ce, int cf, int cg, int ch, int ci, int cj, int ck, int cl, int cm, int cn, int co, int cp, int cq, int cr, int cs, int ct, int cu, int cv, int cw, int cx, int cy, int cz, int da, int db, int dc, int dd, int de, int df, int dg, int dh, int di, int dj, int dk, int dl, int dm, int dn, int _do, int dp, int dq, int dr, int ds, int dt, int du, int dv, int dw, int dx, int dy, int dz, /* Snip 100 lines */ int ya, int yb, int yc, int yd, int ye, int yf, int yg, int yh, int yi, int yj, int yk, int yl, int ym, int yn, int yo, int yp, int yq, int yr, int ys, int yt, int yu, int yv, int yw, int yx, int yy, int yz, int za, int zb, int zc, int zd, int ze, int zf, int zg, int zh, int zi, int zj, int zk, int zl, int zm, int zn, int zo, int zp, int zq, int zr, int zs, int zt, int zu, int zv, int zw, int zx, int zy, int zz) { return a+b+c+d+e+f+g+h+i+j+k+l+m+n+o+p+q+r+s+t+u+v+w+x+y+z+ aa+ab+ac+ad+ae+af+ag+ah+ai+aj+ak+al+am+an+ao+ap+aq+ar+_as+at+au+av+aw+ax+ay+az+ ba+bb+bc+bd+be+bf+bg+bh+bi+bj+bk+bl+bm+bn+bo+bp+bq+br+bs+bt+bu+bv+bw+bx+by+bz+ ca+cb+cc+cd+ce+cf+cg+ch+ci+cj+ck+cl+cm+cn+co+cp+cq+cr+cs+ct+cu+cv+cw+cx+cy+cz+ da+db+dc+dd+de+df+dg+dh+di+dj+dk+dl+dm+dn+_do+dp+dq+dr+ds+dt+du+dv+dw+dx+dy+dz+ /* Snip 20 lines */ ya+yb+yc+yd+ye+yf+yg+yh+yi+yj+yk+yl+ym+yn+yo+yp+yq+yr+ys+yt+yu+yv+yw+yx+yy+yz+ za+zb+zc+zd+ze+zf+zg+zh+zi+zj+zk+zl+zm+zn+zo+zp+zq+zr+zs+zt+zu+zv+zw+zx+zy+zz; }

Brought to you by the Non-WTF Job Board:

This is the version of Saved By The Burst (originally published November 20, 2006) that was included as part of the IASA's Dependency Identification and Management collection.

One of the more popular ideas from the dot-com days was to create a Virtual Currency Exchange. It's hard to say why. Perhaps their founders believed that merchants and consumers really needed yet another “layer of abstraction” in e-commerce. Perhaps they thought that merchants would sign up in droves when they heard about the opportunity to give away another percentage of each sale to another middleman. Whatever the reason, consumers avoided the dizzying medley of incompatible exchanges, leaving only a single VCE contender (PayPal) to remain after the dot-com bubble (and all of the “real” money behind it) dried up.

A Perfect Partner

Diez B. Roggisch's worked for a large travel agency that did a lot of third-party hotel reservations. It was a fairly simple process: the agency accepted their customer's credit card number; they then contacted the hotel and reserved a room with it; and after the customer's stay, the hotel billed the credit card and gave the travel agency a referral bonus.

At the time, small agencies did third-party reservations manually with phone calls, paperwork, and fax machines, while larger agencies (like Diez's) had their software dial directly into the hotel's reservation system. The process was tried, true, and just about as old as credit cards themselves. And therein lay the problem: it wasn’t new.

The company’s board of directors had been drooling over the dot-com pie and knew that the only way they could get a slice of that was by doing “something new” in the world of travel. Something online. Something e-. Maybe even something i-.

The Chief Technology Evangelist

To help the company transition into a dot-com, the board brought a new executive aboard. Though his official title was Chief Information Officer, he preferred being called the Chief Technology Evangelist. As he put it, he was always on top of technology. From mastering that Heathkit he got at eleven to hacking away in that new thing called “Java,” the CTE embraced the newest technology in all of his endeavors, and nothing was any newer than “virtual” currency in the dot-com era.

The Chief Technology Evangelist searched high and low for the best Virtual Currency Exchange in the market. He eventually settled on one that functioned like all the rest did at the time: when a customer was ready to make an online purchase, she would be transferred to the VCE website to complete the transaction. This final step involved giving real money (via credit card) to the exchange, receiving virtual money in return, and then explicitly giving that virtual money to the travel agency. Later the travel agency would then, in turn, take that virtual money and request that a check for real money be issued so they could pay the hotels. In theory, the idea sounded completely unnecessary and a rather obnoxious way to interrupt the customer's shopping experience. That was okay: this was exactly what any online travel agency would have needed.

Designing the Impossible

As Lead Software Architect, Diez was involved with the company’s dot-com initiative from its inception. Though he liked the Chief Technology Evangelist personally, he balked at his idea of using a Virtual Current Exchange. It was simply unnecessary, Diez argued, and didn’t fit into their business model at all. And then he crossed the line; he said it would be impossible to do.

The Chief Technology Evangelist didn’t like the “I”-word. It was like a cross between nails-on-a-chalkboard and a 6x6x6 Rubik’s cube. Words that start with lowercase “i"—iBank, iThermostat, iMusic, iEtcetera—those were good. But this I-word? Sacrilege. Everyone knew that with technology, all things were possible. All the more reason to push ahead.

The Chief Technology Officer implored Diez to go back to the drawing board and find a way to make it happen. No matter how hard he tried though, Diez just couldn’t see a way around a fundamental problem: hotels require a credit card to reserve a room, and virtual money and credit cards operate on two totally different planes of existence.

Diez reported back that there was simply no way they could use a Virtual Currency Exchange. He even suggested an alternative route: accept the credit card information directly from the customer then pass it to the hotels. It was simple and the same way their business was already operating off-line. But the CTE would have none of this; instead, he offered his own solution to the problem: reserve hotel rooms in their name with the travel agency’s own corporate credit card. Though Diez agreed this solution would “work,” his spirit sank once he realized the layers of complexity this would introduce. Building this system was going to be anything but easy.

The Rise of the Dependencies

The main reason that hotels are tied to credit cards is because of the highly profitable class of services called up-charges. An up-charge can be anything from that 75¢ local phone call to the $8 bottle of water consumed from the mini-fridge. Without a credit card on file, hotels would have a pretty difficult time collecting those extra few dollars after checkout, especially from those guests who deem the up-charges a bit too excessive. Which happens to be all of them. It would be almost impossible for a travel agency to collect excessive up-charges via a Virtual Currency Exchange days after the stay. Almost impossible.

The board was a bit wary of the Chief Technology Evangelist’s strategy; it just seemed too complicated. But the CTE’s argued that technology and collections agencies could overcome any complications and that this was the only way to vault them into the already-booming world of dot-com. He also pointed out that customers would be prepaying for their stay, thereby allowing the travel agency to collect interest before paying their own credit card bill, which could make up for any losses and perhaps even earn them a little extra profit. Plus, with their level of credit card volume, they would be able to negotiate a cash-back deal similar to those associated with consumer credit cards. That was all the board needed to hear: they gave the CTE the go-head and ponied-up enough financing to complete the system that, by then, had ballooned to an estimated 100,000-hour beast of a project.

The Fall of Dot-Com

Halfway through development, the dot-com bubble burst and eventually claimed as victim the Virtual Currency Exchange that the Chief Technology Evangelist had partnered with.

Although Diez designed the system so that it could interface with a different Virtual Currency Exchange, the entire workflow was dependent on the virtual money paradigm, which had now largely become bankrupt. There were talks about partnering with the one remaining VCE, but those talks went nowhere; it was painfully obvious that consumers were perfectly content with using real money for online transactions. That meant credit cards, not Virtual Current Exchanges.

In the end, the board decided to cut their losses and cancel the project. The system that cost then several million dollars to develop was completely scrapped; they didn’t even bother trying to find a buyer for its remains. The Chief Technology Evangelist resigned (at the board’s “recommendation”) and the cadre of human processors and customer service representatives brought on to handle the expected upsurge in business were all let go. The board’s last action on the matter was to order that their website be taken down; they were to have absolutely nothing to do with that whole internet nonsense.

Five years later, the company has finally recovered from their extensive losses. They have even lifted their embargo on the internet and put their modest, brochure of a website back up. And that's a good thing, too. They are just in time for Web 2.0.

Brought to you by the Non-WTF Job Board:

It never ceases to amaze me the lengths that certain programmers will go to solve the simplest of problems. Like, say, negation.

When "n * (-1)" Won't do ...

Originally published on August 12, 2004.

... just come up with an overly complicated function that achieves the same thing, like this one discovered by "mightydog":

There's More Than One Way To Neg. A Num.

Originally published on November 15, 2005.

Programmers have an innate desire to reinvent, over-engineer, and over-complicate things. I've always believed it's a result of the "fun" gap between what's needed (a simple HTML form with a generic FormMail.asp script) and what's challenging (a 12-layer Service Oriented Architecture involving a XML-configurable wrappers for every framework class, HtmlFormConfigurationHandlerProviders, FormEmailManagers, a FormEmailManagerFactory, a FormEmailManagerFactoryFactory, and so on).

The good programmer will fight his urge to build The Greatest System Ever Built, instead providing a more modest solution. Dwayne's colleague, on the other hand, will make sure that nothing is as simple as multiplying by -1 ... Dwayne

 

//function performs the negation for integer. public int getNegate(int n) { string strBin = ConvertToBin(n); string strNegation = ""; int c,count,i; int nDec; if (strBin.Length != 8) for(i=strBin.Length; i<8; i++) strBin = "0" + strBin; count = 1; while (count <= strBin.Length) { c = Convert.ToInt32(strBin.Substring(count-1,1)); if (c == 1) strNegation = strNegation + "0"; else strNegation = strNegation + "1"; count = count + 1; } nDec = ConvertDecimal(strNegation); return nDec; }

Brought to you by the Non-WTF Job Board:

With 2008 winding to a close, here's a question to you: what was noteworthy about 2008 as far as programming languages were concerned? To paraphrase Ehud, on topic are notable news about PLT research (direction, fads, major results) (2) notable news about programming languages (whether about specific languages, or about families etc.) and (3) notable news about industrial use of languages/language-inspired techniques (adoption, popularity).

While we're at it, let's score the predictions made at the beginning of the year and laugh at how young and naive we once were or at least make excuses about why our foretelling didn't quite pan out as predicted.

A Secure and Well-Ventilated Location was originally published on January 9th, 2007

A major part of Don Q's job is to fly out to construction sites and setup their computer network. Weeks before doing this, Don meets with the project manager to make sure that every one knows what needs to be done and how to do it. It's up to the project manager to make sure that the workstations, server, cabling, power, etc. are all in place before Don arrives, and Don makes sure to give concise but not insultingly-simple instructions on how to accomplish that. One of these tasks is simply: install the server in a secure and well-ventilated location.

Don had dolled out that task to construction project managers countless times, and they had always complied without issue: the server was installed in a secure and well-ventilated location. After all, where else would one put a $15,000+ computer running $100,000+ software to manage a $50,000,000+ construction project? When Don arrived at one particular location to setup the network, he discovered that the project manager had a slightly different interpretation: The Men's Room.

Of course, it wasn't just any men's room; it was the men's room in a construction site trailer where various construction workers trudged in all day long to do their nasty business. What's not visible in the photographs is the 3" hole in the floor where network cabling exits the trailer. Being that it's unsealed, a nice cold draft, along with a small family of mice, have no problem getting through. Also not visible in the photograph is the mice turds littered across the men's room floor.

Even after confronted and explained the issues with this, the project manager saw no problem with this set up. Fortunately, Don was able to convince the higher-ups and get the server moved to a more suitable location.

Brought to you by the Non-WTF Job Board:

Bunker Buster was originally published on Febuary 19, 2007.

“Hope I’m not waking ya up, but we need a huge favor.” It was just past dawn on a Saturday morning, and Jack’s new boss was on the line. “We need you to replace some servers for the portal system.”

Earlier that week, Jack started working as a subcontractor to maintain and “fix up” a certain government agency’s web portal that was used by thousands of branch offices around the country. Jack’s boss, a tech-savvy businessman who had mostly setup the web portal by himself, was training users and demonstrating the product, and noticed they had a serious problem: their system was running ridiculously slow.

“Okay,” Jack answered still half-asleep, “where should I pick up the servers? Are they at the office?”

Up until this point, Jack had never been to the hosting facility nor had he seen the equipment they were using. It was seven-figure government contract, so he figured they had all the normal stuff: load balancer, redundant power, failover servers, and all that other stuff.

“Well,” the boss replied, “I was hoping you’d be able to head down to CompUSA and build us one.”

Jack wasn’t quite sure how to respond. Sure, he’s pieced together a few computers in his day, but how exactly does one build a server out of consumer-grade components? Before he could even reply, Jack’s boss said “I’ll pay you double.”

Not to be one to turn down money waved in his face, Jack agreed and headed down to CompUSA to build the best computer money could buy. And as it turned out, that was the Ultra Super Extreme 7000XL Deluxe Gamer setup minus the gaming video card. It had a four-disk RAID 0+1 array with screaming SATA drives, plenty of RAM, and lots of blinking neon lights, all which were visible through the cutout biohazard symbol on the side of the case.

After configuring the server, Jack called up his boss and told him that he was ready to take it to the hosting facility. All he needed was instructions on how to get there.

“The facility is about forty minutes from the office,” Jack’s boss replied, “it’s in the middle of a forest outside Conroe. I’ll call them up and have someone meet you at a gas station off of I45.”

The gas station meetup wasn’t arranged for dramatic effect or to protect the location of the facility. The hosting facility was truly in the middle of nowhere. It was only reachable by unpaved access roads, and none of those were on any maps.

Shortly after dusk, Jack arrived at the gas station and was greeted by a fellow wearing dark sunglasses and a black suit. Standing in front of black Chevy Suburban with tinted windows, he introduced himself simply as, “The President.” No, it couldn’t get any more cliché than that.

The President took a look at Jack’s car and said “we better take mine.” They loaded the server up and headed down an unlit county road to the hosting facility. A few miles down, The President turned down an unmarked access road and proceeded to drive for another few minutes through the dense forest. And then all of a sudden, they were in a small, open grass field with two small, concrete buildings and a three-car parking lot.

As they got out of the car, Jack noticed that one of the buildings had metal plates over the windows with small, rectangular holes cut out of the center. They were machine gun ports.

Jack and The President headed to the other building. Inside, it was no more than a small room (about the size of a one car garage) with an elevator at one end. The President walked towards the elevator and placed his palm on the scan pad. The unmistakable creak of a climbing elevator filled the room. Jack couldn’t resist, “what, no retinal scan?”

The President smirked. As they waited for the elevator, he told Jack about the facility. It was a decommissioned military bunker with a 40,000-square foot, two-story underground facility. They had air scrubbers, generators, food, water -- enough supplies to last completely cut off from the world for at least two months. They even had decontamination showers and holding cells. Yes, holding cells.

They descended down into the bowels of the facility and headed through a half-renovated hallway towards a small room that Jack’s boss was renting. It was mostly empty and had cables and miscellaneous parts strewn across the floor. “Let me know if you need anything,” The President said, leaving Jack and the new server alone in the room.

In the corner of the room was the portal system’s rack. It was about five feet tall and had a second-hand monitor sitting on a mounted 1U-hub. Below the hub sat a shelf filled with five or six consumer-grade routers all patched together. Below the routers sat a Compaq Presario desktop computer. It was jammed in and sat at an angle, as it couldn’t fit horizontally or vertically in the rack. Below the desktop were the two portal servers, one to run the database and one to run the web application.

The portal servers were a generic brand of Shuttle PCs: little white cube computers that have room for a CDROM, floppy drive, and maybe a single PCI card. They’re generally what you’d put in your living room as a media PC or what you’d buy for Grandma to check her email. They’re not generally what you’d expect to find in a datacenter running a large government portal.

It took Jack a little while to figure out exactly how to plug in the new server he built. From what he gathered, the hosting facility’s feed went directly into the hub, and each router was plugged in to one of the hub’s port. The servers were then plugged into the routers.

Whomever set up the servers must have had a hard time figuring out how to get the web server open to the “outside” while still being able to talk to the database server on the “inside.” So, he simply assigned both of them a routable IP address and exposed them to the “outside.” It didn’t help that all of the administrator passwords were kept as their default.

No matter, Jack was able to get it working and, after leaving the hosting facility, immediately called up his boss to explain the lax security situation. His boss had a rather odd explanation:

I know the security isn’t perfect, but this is an area we need to tread lightly on. The guy we’ve hired to maintain our servers has some mental issues and doesn’t respond well to criticism or working on Saturdays. I’d rather not upset him now, but I promise, we’ll get there soon.

Three months later, the servers are still completely unsecured and open to the Internet. One of them is currently participating as a zombie in a bot-net. And they’re still housed in one of the world’s most secure underground datacenters.

Brought to you by the Non-WTF Job Board:

Now That's A Neat Trick was originally published on September 9th, 2006.

Brian was excited to join an organization that boasts managing data for "93% of all health care facility inspections." That's important stuff, after all: inspections are essential to maintaining the integrity of facilities and can make the difference between life and death of its patients. But after eight months working there, Brian now looks for facilities in the "other 7%" for his personal health care needs.

While most companies measure the quality of their product with the ratio of bugs to lines-of-code, Brian's employer uses their own, unique metric called The Cleverness Scale. Utilized primary by upper management (also known as, the Original Coders of the system), code and design techniques are rated as follows:

1 - The Shrug-- Gets the job done, efficient, all that; but just plain boring.
2 - The Nod -- Creative, as in, using a single 32-bit integer variable to store two 16-bit integers.
3 - The Raised Left-Eyebrow -- Very creative, as in, combining all function arguments into a single pipe-delimited string
4 - The "Now That's a Neat Trick!" -- The ultimate in cleverness, as in, code that rewrites itself but maintains the original using generated comments

Despite this scale, most of the developers have actually learned to write code that scores only a 1; it turns out that there's some sort of relationship between clever code and incredibly painful maintenance. Actually, the real problem is when the CTO decides to "take the day off" and join the developers to "show 'em how it's done."

Admittedly, the CTO is able to crush objects and goals in a matter of hours. Granted, his solutions are chock full of bugs, implement a fraction of the requirements, and are generally completely unusable. But as this next example shows, his code always employs "A Neat Trick" ...

$sql = " SELECT CONCAT( '<a href=\"javascript:;\" onclick=\"window.open' + '(\'site_info.php?siteid=' ,site_id ,'\',\'siteW',\'height=480,width=550,status=no,' + 'toolbar=no,menubar=no,location=no,scrollbars=yes,' + 'resizable=yes\');\" class=\"siteInfoLink\">' ,DATE_FORMAT(insp_date, '%c/%e/%Y') ,'</a>' ) AS `Inspection Date` ,insp_date AS `_realInspectionDate` ,CONCAT( '<NOBR>' ,DATE_FORMAT(insp_date, '%H:%i') ,'</NOBR>' ) AS `Inspection Time` ,insp_date AS `_realInspectionTime` ,insp_requestor AS `Requestor` ,insp_originator AS `Originated By` ,site_status AS `Status` ,site_loc AS `Location` ,IF (job_priority IN ('Important', 'ASAP') ,job_priority ,'' ) AS `Job Notes` ,insp_type AS `Inspection Type` ,IF (job_priority = 'Important' ,'#990000' ,IF (o_priority='ASAP' ,'#330000' ,'#000000' ) ) AS `_rowColor` FROM tbl_site_inspections INNER JOIN tbl_sites ON site_id = insp_site_id ";

The CTO managed to avoid the Stored Procedures, the Data Access components, the Business Logic components, and smooch SQL, PHP, HTML, and JavaScript into a single statement. A neat trick indeed.

Note to RSS subscribers: to save on bandwidth, I'm moving the RSS Feed to FeedBurner. Please let me know if you experience any disruptions.

Brought to you by the Non-WTF Job Board:

Diary of a Third-Class Programmer was originally published on September 27th, 2007.

Most first-class (consultants & employees) and second-class (contractors) programmers don’t realize that there’s an entire class of programmers below them. These programmers – the third-class programmers – work in Developer Purgatory with far less responsibilities, no latitude to make any decision at any level, and always get assigned the “dirty work.” In fact, some even consider it an act of charity to call these poor folks “programmers” at all.

Most third-class programmers are third-class for a reason. There is simply no way they could ever get a job anywhere else. Occasionally, recent graduates get sucked in to Developer Purgatory by the allure of a “real programming job,” never realizing the stigma that goes along with it. Andrew is one of these poor souls, desperately trying to get out. And this is his diary.

May-20: First day on the job! I guess the department is officially titled “Internet Administration” (whatever that means), and has some fun people it. Not at all what I expected from The Bank!

May-22: Met Diane today, she just started working on the “Repo” database project. She said she’ll definitely be asking for my help on it soon.

May-23: I met April today. She seemed nice, though it was a bit odd that her entire desk was covered with Pokemon merchandise and stickers. Her other passion, so she said, was playing First Person Shooter games online. Other than that, it’s been a slow first few days with nothing to do.

May-24: Met Steven today. He’s an ASP programmer for some internal sites. Later in the afternoon, he stopped by cube, worried. To quote: “You're going to have to help me on this, because I know absolutely nothing about ASP.”

May-27: Diane said she might need some help on the “Repo” database. She’ll let me know. I don’t have anything else going on.

Jun-04: Chatted with April (Pokemon girl) about Unreal Tournament. She’s in a “clan.” It is her mission, apparently, to run out into the middle of an open area and get shot down so her teammates could spot the enemy.

Jun-18: Don (our manager) got really peeved today. We were playing foursquare between cubes again, and the ball bounced twice in his. He should have been out, but he kept insisting that a desk-bounce doesn’t count as a bounce. Anyway, he stormed out of the office, saying he’ll never play again, and went home early.

July-08: Diane asked if I had free time to help her on the “Repo” database. I did, she said she’ll let me know soon how I can help.

July-18: Steven was really late today. Apparently, he was getting onto the train and his shoe got stuck in the door and left behind on the platform. He had to go back and fetch it.

Aug-20: Diane asked, again, if could help her on the “Repo” database. I said yes, again.

Aug-26: New guy, Colt, started today, transferred from Investment Banking. He moved here because he felt he was stronger in this area.

Aug-27: Colt, as it turns out, has an unhealthy obsession with firearms. Who keeps pictures of their guns in their wallet!?

Sep-16: Diane snapped at me today when I asked how she was doing. To quote: “I’m too busy working on Repo to chit-chat!”

Sep-17: Diane apologized for yelling at me. She’s been under a lot of stress with Repo.

Oct-09: Colt broke my keyboard today. I complimented his new shoes. “Thanks,” he replied. And then, he proceeded to grab a nearby glass of water and tipped it over onto my desk. This caused a large puddle of water that started dripping onto the keyboard. He then lifted up one of his legs, and put his shoe on the desk, in the middle of the water. He then grabbed his ankle, tugging at it, and said, “good grip in the wet too.” And then he walked away.

Oct-14: Another new guy, David, started today. He virtually wanted to move away from mainframes and basically move into web development. He basically got a lot more money than I did because he'd virtually been working at the bank longer. Basically though, he wasn't very good at web development. Oh, and he virtually spoke like this all the time.

Oct-22: Finally got to see the “Repo” database. It was literally three tables in an Access database and had a “form wizard” form called Form1. The form didn’t do anything. This is what Diane has been working on for six months.

Nov-04: Don asked me to review David’s code, as it was his first coding on a full-color screen. He had been working on a Budget Planner calculator that had lots of amounts the user could enter. Food, electricity, gas, etc. Next to each of these items was a drop down box with “Weekly,” “Monthly,” “Annually.”

Nov-05: I took a look at David’s code. I suggested that, instead of having twenty odd “Weekly,” “Monthly,” “Yearly” options throughout the page, he should put them into an array and reference that. He was happy to learn a better way of doing things.

Nov-08: David showed me his revised code. In the code now, there was not just one array, but about 20 arrays declared, each array containing “Weekly,” “Monthly,” and “Yearly,” with each drop down box pointing to a different array. I told him he should just be using one array, as they are all the same.

Nov-13: David let me know that he fixed the code again, so I took a look. This is exactly what I found:

arrayOne = "Weekly", "Monthly", "Yearly" arrayTwo = arrayOne arrayThree = arrayOne arrayFour = arrayOne ...

Brought to you by the Non-WTF Job Board:

Journey to the Center of the Database was originally published on June 5th, 2006.

It'll be a long while before Donnie will forget about March 5th, 2004. Early that morning (during one of the ungodly hours), Donnie received his first off-hours support call. Naturally, one of the "mission critical" applications (more specifically, one that Donnie knew only by name) was completely broken and needed to be fixed right away. With the primary and secondary support guys unavailable, Donnie was on his own.

After waking up and reading the sparse documentation available, Donnie looked at the errors and noticed that most seemed to originate in the database. A bit more digging and Donnie found that the problem was at the apparent center of the database, the Date Table:

As it turns out, the system "abstracted" out dates into a separate table and every table requiring a date field used the DateId instead of the real date. The application didn't cope so well when the Date Table ran out of dates. Donnie inserted ten year's worth of dates into the Date Table and, after filing his report, the primary support guy thanked him and mentioned "yeah, this happened last year; we probably should have put more than a year's worth in there."

Brought to you by the Non-WTF Job Board:

The next Flex/AIR workshop with James Ward and myself will be February 25-27, 2009, Crested Butte, Colorado, This is right before the JavaPosse Roundup, so you can get the most out of your airline ticket.

Because of a conflict with the SD conference, we moved the Roundup to March 3-6.

They goad you into "Standard Edition" but once you change, you can't go back. No matter how bad it is. Change at your peril.

Publicly-held corporations ostensibly serve the people they sell products to, but are required by law to actually serve their shareholders. Privately-held companies only serve the customer.

James Ward (Adobe evangelist for Flex) and I have coauthored a book to rapidly bring programmers into the world of Flex. Our primary intent was to make the book small (only 140 pages), with short, pithy chapters, to lower the hurdles to learning this powerful GUI system.