Uncovering the truth behind “50% of teens post senstive information”

Last week we saw a plethora of New Zealand headlines bemoaning the poor behaviour of ‘kids today’ – this time in how they handle their sensitive information online.

The punchline was that one in two students had posted sensitive information about themselves online in the past year.

TVNZ went with Half of NZ teens post sensitive info online while NZHerald went with Nearly half of Kiwi teens post sensitive info online – both remarkably similar to the original press release.

Some went further though – and the three best articles that I read were from:

1. The venerable ODT – with Teens lax over online security a very well written article that included a local angle and even a call to a local academic.
2. The Dominion Post with Internet’s effects may be taught – an interesting angle, and with only passing reference to the survey. Well done to the Dom Post and Greer McDonald.
3. TV3, with some new news Policing unit to monitor internet for criminal activity which probably means bald 50 year old men masquerading as 15 year old girls are going to start flirting with me online.

The articles all stemmed from preliminary findings published by PhD student, and research manager at Netsafe – John Fenaughty.

Colour me skeptical. Indeed I am frighteningly skeptical about any headlines that say “the youth of today are….” as I remember all too well that the youth of my day were actually pretty on to it.

I suspect (and from what I see, know) that the youth of today are much better at figuring out what they can and cannot put online than their older peers. In particular I see that horribly inept early Bebo pages and youthful utterances are increasingly becoming the norm, and employers and voters of the future will accept it as such.

But I was also concerned that the survey itself was a bit of a half baked scaremongering exercise. So I decided to dig into it a little. The Netsafe website was useless – but I did find the press release after a fellow twitterer shared the link.

So I called (there was no email address) John Fenaughty and left a message. He got back to me very quickly on email, and I posed him 12 questions – cunningly displaying them as 10:

1: How was the final respondent group selected, including response rates etc? How did you avoid bias across the multiple dimensions?
2: What were the exact dates of data collection?
3: What were the actual survey questions used to derive these answers? especially the “wouldn’t want to find” part.
4: How were the questions asked? – e.g interviews, filled out by students etc.
5: What was the age (or school class) distribution of the responding students?
6: What percentage of students didn’t use the internet? were they included in the survey?
7: How was the “one out of two” figure derived? – can you provide a break down (crosstab) that shows the combination of sensitive information provided?
6: What is the breakdown of age (or school class) versus provision of the four sensitive informations identified in the press release?
7: What other questions were in the survey?
8: will you be making all of the coded source data available?
9: How do you define cyber bullying?
10: What convictions for cyber bullying have there been in NZ?

Quietly readying myself for a nice evisceration of the study, I noted that his reply wasn’t instant – and perhaps wasn’t ever going to come. I mulled on the state of research these days, but eventually I did get a reply almost a day later.*

It was rigorous. In fact it was an excellent reply, and I am posting it in full beneath the fold.

John sent me back complete answers to all of my questions, and satisfied my greater concerns about the study. Simply put – he is doing the best he can within the constraints he has been dealt.

I noted to myself that this is just the sort of evidence that you would want to see from a PhD student, especially as a PHD needs to be defended in front of a committee. Having sat on one of those committees before (we had to say no in the end) it is a grueling exercise for the student, but with the quality of this response John is demonstrating that he will be ready.

If you read the reply, ask yourself whether any research that you conduct or read about can be answered just as well. Are your questions tested? Are all ethical grounds covered off? Are the samples truly statistical and non-biased? and so on.

So well done John – Not only have I deleted most of the blog post I’d drafted, but you’ve even managed to turn it into a “how to respond to questioning bloggers” lesson.

(From John Fenaughty)
Hi Lance.

Thanks for your email and your questions. The data that we released for Privacy Awareness Week was a preliminary look into the data set that I’m going to be doing more work on in the following year. A full report (in addition to the PhD thesis) will be produced covering off all of the specifics by this time next year. In order to “get the data” out there, we are releasing some findings early. I’ll get onto responding to some of your questions below – as you’ll see I’d like to invite you to come in, or call for a chat about some other things too.

1.) The participants in the research were high school students from five schools in NZ. Schools were selected that ranged across a number of dimensions. These dimensions were selected to survey a wide variety of young people in NZ about their experiences online and on mobile. Dimensions included national location, decile rating (a measure of the school community’s social economic status), single sex vs co-ed, and metropolitan vs semi-rural. Schools ranged in decile rating from decile 4 to decile 10. Schools ranged from non-metro to metro to give a spread in terms of urban versus non-urban activity. Locations included South Island semi-rural, Wellington, Central North Island, and Auckland.

The research, being conducted at the University of Auckland, went through UoA ethics review process. This requires schools to consent to research participation. From then, participants under 16 years of age required parental consent to participate in the research, as per UoA requirements. The final response group then includes those students who, if permitted, chose to complete the voluntary activity. Response rates varied across the schools and the samples. The overall response rate was just over 60% across the entire sample, with a range from just over 50% to 90% response rates depending on the school.

Regarding avoiding bias over multiple dimensions – this would be something that would better off being covered by a conversation (the standard issues of social science come into play here around self-selection bias and convenience sampling vs social desirability bias and the pragmatics of conducting research involving school students in NZ).

2.) Data collection over multiple sites was conducted from the third week in August 2007 until the third week in March 2008. Depending on the school schedules data collection was staggered over different schools during this period. Specific data collection periods at each school varied depending on the size of the school and the school resources – on the whole though most data collection was complete within a school week.

3.) The survey is the second part of data collection for the PhD. The first phase involved a series of 8 focus groups around the country at an equally wide variety of schools. The focus groups were the first point of departure for the research and were used, in part, to guide topic and questionnaire development for the survey. After the focus groups, questions were developed which were then subjected to the UoA ethics review process. After receiving ethics approval for the questionnaire, the first set of questions were piloted with 10 teenagers to ascertain that the questions were asking what was desired and were understood by a range of ages (and across young men and women). Following that phase of the research design, small changes to wording were made and the questionnaire was ready to be put to use. The use of the “someone who didn’t like you” was used in this particular quesiton as this helped make the hypothetical audience of public viewers more concrete for some of the younger teenagers [rather than just someone who didn’t know you]. This is in line with theories of adolescent development which demonstrate that younger teenagers may still be operating at a more “concrete” level than the more formal abstract thought that becomes available as teenagers age.

A key focus for this research was also to leave questions open for students to answer as they wanted to, rather than predetermining what we thought was sensitive information.  We did this by including the “anything that you wouldn’t want someone who didn’t like you to find” response. You’ll also see that we employed a similar approach with the cyberbullying item.

The specific question around sensitive information was:

In the past year, have you ever posted any of the following things on a public website or mobile phone chatroom where someone you didn’t already know face to face, OR someone who didn’t like you, could find it?
(like posting it on ‘open’ blog sites, an ‘open’ page on Bebo, YouTube, web forums, chat rooms, etc)?

(Please tick ALL the answers that apply)
– Your mobile phone number
– Your instant messenger ID or email address
– The address of your home
– Your first and last name, OR your first name and a recognizable picture of yourself, AND anything (like pictures, stories, or comments) that you wouldn’t want someone who didn’t like you, to find
– Or… I haven’t posted any of these on a public (or ‘open’) webpage.

4.) The questions for the survey phase of the research were asked in an anonymous survey filled in by students themselves during school time.

5.) It’d be good if we could have a conversation about how much specifics we can share with you and the media, perhaps you might be able to come in (if you’re in Auckland) or we can talk about this on the phone. Researchers face a balance between sharing research findings and managing the original material copyright requirements for academic journal publishing [which require that detailed research results not be published prior to that publication]. If you’d like to come in and meet me and the team that would be great and we can talk to this also. We keep findings released to the public and the media at approximates to enable us to contribute both to the ‘grey’ research literature as the findings are useful and then later to the academic research later increasingly desperately seeks peer reviewed data [the majority of internet data at the moment is ‘grey’ research literature].

With that in mind, the data included:

12 year olds (a handful)
13 year olds (aprox 200)
14 year olds (aprox 350)
15 year olds (aprox 350)
16 year olds (aprox 350)
17 year olds (aprox 300)
18 year olds (aprox 100)
19 year olds (a handful)

6.) As mentioned in point 5 we are not able to provide this specific information at this point. However once the final research report is released we will be in a position to publish the specific data on these points. In vague terms, descriptive analysis reveals many commonalities on each of the types of sensitive information published in public places online across the age groups:

There were similarities (results fell within one standard deviation of the average response rate for that item) between age groups for each of the following:

Posting a mobile phone number: Same amounts for 13, 14, 16, 17 and 18 year olds. 15 year olds were, on average, one standard deviation more likely than the other participants to report posting a mobile phone number.

Posting an email address or instant messenger ID: Same amounts for 14, 15, 16, 17 year olds. 13 year olds were, on average, one standard deviation less likely to report posting an email address or instant messenger ID. 18 year olds were, on average, 2 standard deviations more likely to report posting this information publically than the other age groups.

Posting the address of your home: Same amounts for 13, 14, 15, 16, 17 year olds. 18 year olds were, on average, one standard deviation more likely to report posting their home address than the other age groups.

Posting sensitive info you wouldn’t want others who didn’t like you to find: Same amounts for 13, 14, 15, 17, 18 year olds. 16 year olds were, on average, one standard deviation more likely than the other participants to report posting such information.

On the whole the findings reveal very similar reporting rates for the information across the age groups. In the 24 domains of data sharing (data by age group) there were only 4 domains where results differed by more than one standard deviation, and only one where results differed by two standard deviations. The most significant finding seems to be that disproportionately more 18 year olds post their email address or instant messenger ID online, than the other age groups. Of key interest is the result demonstrating that some participants from within each age grouping report sharing each of these types of information publically.

7.) The survey asked students a range of questions, including:

Their activity in online/onmobile (amount of communicating, researching, gaming, filesharing, trading, publishing, friendships, romance, media consumption, etc.).

Measures of adult help seeking behaviour (how readily and easily they report asking adults for help), social self-efficacy (how well they get along with peers and classmates), and family support (how supportive they report their families to be in helping them with problems and giving them love and support).

Experiences of challenge online/onmobile in the last year, including: bullying, meeting face to face with people first met online/onmobile, consumption of adults only sexual material, exposure to unwanted sexual material, exposure to aggressive sexual solicitation, exposure to other content online/onmobile that made them uncomfortable or upset, downloading [not streaming] music or movies for free via file sharing applications, being scammed, and gambling.

How they felt about the most serious time they experienced each challenge.

What action they took to manage the challenge if it was upsetting.

How well they felt that particular challenge had been ‘sorted out’.

6+7 v2) Re the second set of 6s and 7s:

Only a couple of percent of participants reported not using the internet in our sample. They were included in the one out of two finding (obviously contributing to the not posting sensitive info online side of the equation).

Re. How was the one out of two figure derived?
This figure was derived from a participant reporting that they had posted at least one item of the four forms of sensitive information online. The figure weirdly came out nearly almost perfectly as 50%. Going deeper, just under a third of the entire sample reported having posted more than one of these pieces of sensitive information online in the past year. As noted above, the entire sample includes those students reported not using the internet.
8.) In addition to what I’ve mentioned in point 5, we have not anticipated making the data publically available. Publically available data requires additional ethical UoA Human Subjects Ethics Commitee approvals and processes. After completion of publications, researchers wishing to conduct analyses that have not already been conducted would be welcome to contact the research team with their analysis requests.

9.) In the focus groups we examined the language around cyberbullying. There is a lot of variety in how young people understand cyberbullying. We decided to make the initial cyberbullying question relatively open ended to collect this range of responses before offering more specific examples in the following question. Additionally, the focus groups revealed that many young people trade what may, to outsiders, be perceived as bullying comments online. Asking then if someone had received text messages that included swear words or threats can over represent bullying by subsuming now normal teenage online banter as cyberbullying. There were two cyberbullying strands – one for bullying online and one for bullying onmobile. Both were relatively similar, they just assessed each modality of cyberbullying separately as focus group participants had mentioned that bullying on mobile was more significant than bullying online. The mobile bullying question was:

In the past year has someone ever tried to use a mobile phone to bully or be mean and hurtful to you?

If they responded yes to that the next question asked:

Thinking of the most serious time in the past year that someone tried to bully, or be mean, or hurtful to you, on a mobile phone,

What did they do?
(Please tick ALL the answers that apply)

They used a mobile phone to:

Please tick if this happened
…say or text mean, hurtful, or nasty things to you
…to spread rumors about you (even if they were untrue)
…not let you talk, text, or be friends with them (like they ignored you)
…send you scary or disgusting pictures and videos
…send mean or embarrassing pictures or videos of you to other people
…threaten to hurt you physically (like txting to say they were going to get you)
…threaten to tell other people embarrassing things about you
…threaten to damage and hurt someone or something that you cared about
…do something else mean or hurtful on a mobile phone
And…they did OTHER mean and hurtful things NOT USING a mobile phone

10.) Jani at NetSafe has prepared some stuff for you about the legal issues with cyberbullying. [I didn’t receive this – LW] Hopefully they shed some light on the legalities? Personally I’m not aware of a conviction, although I’ve heard about things floating round in relation to abuse of protection orders and last year an 18 year old woman was charged for hacking into a younger teen’s bebo site with the implication that this was a cyberbullying scenario – though I’m not sure – the case was in Hastings. Jani highlighted that harrassment online can involve the legal system when it intersects with the following acts of law:

Films, Videos, Publications and their Classifications Act 1993
Protection orders – Domestic Violence Act 1992 part 2
Restraining Orders- Harassment Act 1997
Telecommunications Act 2001
Hacking and Denial of service attack – The Crimes Act 1961 Section 250
Crimes involving computers – The Crimes Act 1961 Part 10

Judge David Harvey is a NetSafe member and wrote the following which may be of interest:

Click to access netsafepapers_davidharvey_cyberstalking.pdf

Main points: the Harassment Act 1997, the Domestic Violence Act 1995 and the Telecommunications Act 2001

OK – so that’s pretty much all that there. Re the bias question and the specific data questions it’d be great to chat about that in person or on the phone. <snip>

* I should note that the span of about 20 hours for replying to my questions was very quick considering the nature of the questions, the fact that John was undoubtedly fielding plenty of other questions and the qualiy of the answer.