Big data – what will you use it for?

I was speaking to a colleague today in Australia – he was upset that Telstra  increased his data plan to 50 GB without changing the price. His point was that he only uses 4 GB per month, but Telstra gave no option for him to reduce his cost by reducing his cap – the next step down was a tiny 2 GB per month.

That’s clever pricing by Telstra, but it is also opening them up to any competition, offering, say, 20 GB plans for $20 less per month.

On the other hand I see it as a good thing.  Almost everyone downunder uses the internet with one eye on the usage meter, which constrains our activity. With a much larger cap we can begin to use other services that until now are out of reach.

Three examples that come to mind are:

Dropbox lets you sync your files across computers and devices. I use it, but not as much as I should. I’m pretty sure it added at least 5 GB per month to my use, and at a time when I was mainly using data sticks and tethering.

I only synced a few files – but I have 60GB of songs and about 23 GB of core business files that I’d like to have backed up on the cloud and synced with all devices. I cannot see a way to do so with data caps and the relatively slow speeds we have.

Back to My Mac

Back to my Mac lets you control your mac computer at home from afar. It’s a wonderful feature, but useless on ADSL which has slow upload rates, and with low data caps. I would like to run light computers when I travel and have the big kahuna and server back at home, but for now I carry the home computer with me almost everywhere.

Flickr (and others)

I have a flickr account, but use it almost exclusively for uploading pictures for this blog. I’d really like to have the rest of my phot collecton online, synced and available to all of my devices. I’d make most of the photos private, but thousands would be public.

Sadly I have 71,794 photos and 1,298 movies on iPhoto, which is 120 GB of files. That’s a ludicrously large amount of data to be syncing in a world of ADSL and data caps.

Facebook is making a good case for storing my photos and videos as well, as is YouTube for videos.

What other data intensive services are we missing out on? I’d really like to read your answers to this.

One answer is of course ‘the future.’  YouTube only turned up in 2005, and we have no idea what the big data applications will be in 2020. We certainly won’t be developing any downunder unless we are able to fully enjoy unconstrained, uncapped and unfiltered internet.

Published by Lance Wiggs


5 replies on “Big data – what will you use it for?”

  1. 1 – Streaming video to my TV. Being able to watch a sporting event anywhere in the world that Sky TV might not cover.
    2- Video blogs in HD
    3- I find a few hours of video SKype rapidly blows my limit.

    Looking forward to not even thinking about how large some files are getting for my usage.


  2. A couple of points in no real order:

    1. Cap is not the only problem. “Big data” in the future is as much about not having a cap as it is about raw speed. I could have a 20GB cap now and not enjoy YouTube if my speed is 128kps. What sort of speeds is Pacific Fibre offering? Your 120GB sync example would fall down in NZ even if we had no caps due to the length of time it would take to download that data (or, heaven forbid, upload it in the first place!)

    2. The argument that somehow NZers can’t develop these solutions because of our woeful caps is pretty laughable. It costs a few bucks and you can host your solutions on a cloud offering overseas. Basically if you have enough bandwidth to have a remote connection to a server (few 10’s of kb/s) you could build a big data application. I really wish you’d stop using this argument as it’s pretty weak and only going to believed by Mum & Dads who know nothing about IT (Poor Johnny could never build a significant Internet business because he doesn’t have a fat data plan).

    You raise some valid points, but they’re valid to consumers of big data. Your points aren’t overly valid to people wanting to build future big data applications.



    1. End to end capacity is the constraint here, and there are several bottlenecks. For example ISPs have to divide their purchased international capacity amongst all of their customers, and that contention has to happen as the prices are high. Prices have come down a lot recently (I wonder why) though.

      It’s not that NZ developers cannot develop these solutions, it’s that there is not a ready supply of NZ consumers both demanding the new new thing and being consumers that turn into developers. Meanwhile would you be able to compete effectively as a develop while on a dial-up modem?



      1. As mentioned, I applaud adding capacity, it is needed. Improving the supply is typically evolutionary in the consumers eyes, but somebody has to do it so please don’t take this as a critique of the overall plan.


        >> Meanwhile would you be able to compete effectively as a develop while on a dial-up modem?

        You know what makes an argument weak? When you give a contrived example. I’m not on a dial up modem, haven’t been for almost 10 years. It would be like you saying today I couldn’t compete with a car if I had to do so on foot. Of course I couldn’t, but it doesn’t make your example make any sense when I already own a car.

        While our products aren’t big data products, my company which is based in NZ makes 99% of product sales to other countries, all delivered electronically and we’re successful doing it. Our website is hosted in Canada, our files hosted on Amazon’s CDN… we can compete quite happily and we out innovate our much larger competitors constantly. Shockingly this somehow happened without big data caps.

        Innovative and/or successful technology companies do not need to be big data companies, and even if they are they would be foolish to host in NZ because unless you’ve solved the speed of light problem, latency would suck for anyone outside NZ anyway.

        Lets just be honest about why big data is great – backups/sharing and video. The internet is fast becoming the defacto standard in information transfer. We don’t need phone lines we have voip. We don’t need cable, we have IPTV. We don’t need faxes, we have email. The internet will replace most mechanisms for data transfer and become even more of a utility than it already is thanks to the flexibility it offers.

        So lets stop saying it’s going to massively improve innovation – it’s not, innovators innovate despite the challenges which is why they’re innovators.

        The rise of big data means consumers have higher quality experiences. The product that will likely swamp your pipe is going to be the rise of IPTV. Eventually everyone will have something akin to an Apple TV that streams movies and television globally and that’s going to chew up a ridiculous amount of data (as part of that, I believe it’s why none of those products have caught on so much yet – very poor consumer experience due to the data but we are at the start of the curve and are about to jump the chasm).

        All that said, re-read the first paragraph :-)



  3. Lance – my frustration is that we constantly seem to miss out on the really nice streaming services that the rest of the world (or at least the US and the UK!) seem to get – primarily in the consumer arena.
    I would be really interested in Google TV, except here in NZ we won’t be able to get the things I would actually want it for such as Netflix. The spotify/pandora services are other nice-to-have’s, although grooveshark is pretty good.
    When I do video editing, I would love to use something like carbonite to backup to the cloud, but again – I have the same concerns as you. It doesn’t take a lot of HD video projects to blow that cap out of the water, Plus of course there is my standard (somewhat heavy) internet usage due to my profession.

    The one thing I did do, was back up my pictures. I actually went with Picasaweb in the end, because I like the client (and iphoto is awful with a lot of pictures) and because they seem to be developing its capabilities faster than anything else out there. The tipping point was when i realised that i could sync my albums one at a time, which meant that I manually throttled my uploads over about 3 months. Now I have all 60GB of photos on my local machine, on a networked NAS and in the cloud as well, which goes some way towards assuaging my paranoia about losing the whole lot!

    I would definitely like to do the same with my music and have the cap big enough and the speed fast enough that I could stream my own, paid for music anywhere any time. It’ll come, just probably not fast enough for those of us that care.


Comments are closed.