Go back

No. 49 No Grey Areas – Against Sexual Exploitation of Children

Workshop Format. Please click here for a description of available Workshop Session Formats.

Other - Speakers Corner - Break-In Session

Duration of proposed workshop

90 minutes

IGF 2015 subtheme that this workshop falls under

Internet and Human Rights

Description

Fortunately, many international laws prohibit child sexual abuse imagery (CSAI) and these laws have been used by industry leaders to make it much harder to find such content on the Internet. However, sexual exploitation of children often begins with content that doesn’t cross the lines drawn by the law, but is still exploitative and deeply disturbing. Depictions showing children in an unnatural, sexually suggestive posture are spread throughout the Internet and are easy to find. This content damages children’s dignity and privacy. There is also a risk that proliferation of this content normalises the sexualisation of minors and makes people think it is an everyday occurrence. The workshop will address how collaboration of stakeholders from industry, government, research and the welfare sector can set the scene for combating and condemnation of such content. Light will be shed on the so called "Grey Areas of (legal) sexual exploitation of children, and countermeasures will be discussed. Bringing together technical knowledge and analytical research expertise on the one hand with the victim's and perpetrator's perspective on the other will design a new approach to fighting CSAI. Artificial intelligence can be of tremendous value for understanding the concepts of provision of CSAI and posing imagery and comprehending the search strategies of perpetrators regards such imagery. In addition knowledge regards commercial transactions and cash flows will help to develop a strategy of combating and condemnation of CSAI in a broad understanding, i. e. covering also the "Grey Area" of (legal) child sexual exploitation.

Name, stakeholder group, and organizational affiliation of workshop proposal co-organizer(s)

Natasha Jackson (Private Sector) GSMA Mobile Alliance (confirmed)

Has the proposer, or any of the co-organizers, organized an IGF workshop before?

yes

The link to the workshop report

http://www.intgovforum.org/cms/wks2014/index.php/proposal/view_public/154

Subject matter #tags that describe the workshop

#CSAI, #Artificial Intelligence, #Sexualisation of minors, #commercial exploitation of minors, #sexual exploitation of minors

Description of the plan to facilitate discussion amongst speakers, audience members and remote participants

A Speakers Corner Break-In session consists of a number of short statements from speakers each followed by break-ins from the audience done either on-site or remotely.

Within this workshop the following topics will be addressed. As an introduction the question what constitutes the "Grey Area" of (legal) sexual exploitation of children shall be answered by Gitte Jacobsen, a reputable expert from Save the Children Denmark with longstanding expertise in the field. It is mirrored in the diverse landscape of national legal regulation towards the topic that there are huge differences in the judgement of what is considered illegal, this shall be brought into the debate by break-ins from the audience either on-site or remotely.
In the same way the perspective of industry regards provision and search strategies of grey area material, the role of artificial intelligence to detect the material and to analyse search terms and phrases used by perpetrators will be addressed before then the commercial aspects in dissemination and exchange of CSAI and Grey Area material gain the floor. Each of these topics provides for the opportunity for stakeholders from various areas to bring in their position in the break-ins.

Eventually when the perspectives of speakers and participants on-site and remote are presented the scene is set for combating and condemnation of Grey Area material. On this basis the idea of the "Network No Grey Areas on the Internet" shall be spread and new partners may express their interest to join the network on mutuality.

Names and affiliations (stakeholder group, organization) of the participants in the proposed workshop

⇒ Gitte Jakobsen (Civil Society) Save the Children Denmark: Gitte has coined the term "Grey Areas of (Legal) sexual exploitation of children. She has broad expertise in the field of identification of CSAI and the legal situation regards such content in many countries, participation is confirmed (remote participant)
⇒ Thiago Tavares Nunes de Oliveira (Government) Presidente da SaferNet Brasil: Thiago has long-standing experiences in fighting CSAI and in running a hotline for reporting, participation is confirmed
⇒ Marie-Laure Lemineur, ECPAT International, Head of Programme, Combating Sexual Exploitation of Children Online. Marie-Laure has outstanding knowledge on the topic of child safety, participation is confirmed
⇒ Akio Kokubu (private sector) Vice President of the Internet Association Japan / Internet Hotline Center Japan: Akio Kokubu has long-standing experiences in fighting CSAI and in running a hotline for reporting participation is confirmed (remote participant)
⇒ Grégory Mounier (Intergovernmental organization) Europol Cyber Crime Centre: Gregory Mounier has long-standing experiences in fighting CSAI, participation is confirmed
⇒ Marco Pancini (Industry) Google Marco represents the position of the industry partner in the Network No Grey Areas and will explain what is Google's contribution to fight abuse and sexual exploitation of children on the Internet
⇒ Christian Schulze (Technical Community) DFKI – German Institute for Artificial Intelligence: Christian is a well-know expert for his research in artificial intelligence and its application for analysis of content, participation is confirmed (remote participant)
⇒ Jean Paul Nkurunziza (Civil Society) ISOC Burundi, deputy Chair: Jean Paul has broad knowledge on Internet Governance procedures and their relevance for child online safety, participation is confirmed (remote participant)
⇒ Natasha Jackson, (Private Sector) GSMA Mobile Alliance (confirmed)

Name of in-person Moderator(s)

Jutta Croll

Name of Remote Moderator(s)

Katharina Sponholz

Name of Rapporteur(s)

Katharina Sponholz

Description of the proposer's plans for remote participation

The format of a Speakers Corner is perfectly well designed to allow for remote participation. Statements of the speakers are kept very short (5 min.) not to overstretch the attention span of the audience which is especially useful when it comes to remote participation. Time for break-ins from the audience is a little bit longer (7 min.) for each topic thus to enable more than one break-in to each speaker’s slot.

To encourage remote participation broadly and in as much countries around the world as possible the workshop will be announced via the networks of the organisations involved, f. e. the workshop organiser I-KiZ will address about 250 stakeholders internationally. The workshop agenda and questions to be discussed (see. above) will be sent in advance to potentially interested participants. Thus it will be easy for them to follow the debate and prepare for their own break-in.

Background paper

background paper

Agenda

Agenda

Min 0 - 3: Welcome and introduction (Jutta Croll, German Centre for Child Protection on the Internet)

Min 4 – 15: Drawing the picture – What constitutes the "Grey Area" of (legal) sexual exploitation of children? (5 Min speaker Gitte Jakobsen (remote) + 7 Min break-ins from the audience)

Min 16 – 27: Understanding the concepts – Which concepts of provision and search strategies of perpetrators are in use? (5 Min Marco Pancini, Google + 7 Min break-ins from the audience)

Min 28 – 39: Analysing the background – How can artificial intelligence and so called multimedia opinion mining help to analyse search terms, phrases and sentences of perpetrators? (5 Min speaker Christian Schulze, DFKI (remote) + 7 Min break-ins from the audience)

Min 40 – 51: Addressing the commercial aspects – Which commercial transactions and payment measures are supportive to the dissemination and exchange of CSAI, how do they count for Grey Area material? What role do virtual currencies play? And how can the – real and virtual – cash flow be dessicated? (5 Min speaker Grégory Mounier, European Cybercrime Center + 7 Min break-ins from the audience)

Min 52 – 63: Reacting to reports - The role of hotlines (5 Min speaker Akio Kokubu Japanese hotline (remote) + 7 Min break-ins from the audience)

Min 63 – 74: Syntesising the powers – The Message of the Communiqué "No Grey Areas" adopted by child welfare organisations around the world (5 Min speaker Marie-Laure Lemineur, ECPAT International + 7 Min break-ins from the audience)

Min 75 - 90: Setting the scene for condemnation – How can the "Network No Grey Areas on the Internet" be expanded further? (20 Min conclusions)

Key Issues raised (1 sentence per issue):

- Hotline reporting mechanisms to enable users to easily report any incidences of child sexual abuse content and to ensure that such content is assessed appropriately and either forwarded to law enforcement when it is illegal or to platform providers for removal when it might be not illegal but is still of concern
- Notice and take down procedures for Grey Area Imagery although it might not be considered illegal in all countries; respect for the right of freedom of expression balanced with full recognition of the right of the child to be protected
- Huge importance of public awareness for “Grey Area” imagery and countermeasures to be undertaken including the aspect of self-produced sexual content and the necessity not to assign the sole responsibility to young people but to other stakeholders in charge
- Improvement of investigation by analysis of Grey Area images because they provide more useful information due to the carelessness of offenders thus being beneficial for identification of victims and prosecution of perpetrators

Please describe the discussions that took place during the workshop session (3 paragraphs):

A debate was instigated by a break-in comment from Natasha Jackson from the GSMA on the different approaches of access providers and service providers/hosters. Mobile access providers do have hotline reporting mechanisms usually working with the hotlines in their countries to make sure their customers can easily report any incidences of child sexual abuse content. Secondly they have notice and take down procedures within their organisation. But different to service providers / hosters they do not assess content and do not make any decision on the content themselves. There they rely on lists they receive i. e. from law enforcement. With regard to the commercial aspects of “Grey Area” imagery attention must be paid to whether mobile payments might be in use for this purpose.
The role of hotlines was clarified by a break-in comment from Amy Crocker from the INHOPE Foundation. She explained that the national hotlines make their decision how to react on reports on “Grey Area” imagery based on the national legislation. They would also take into account their relationships to industry and law enforcement also. From a child welfare perspective there is huge importance to have a public debate on the “Grey Area”.
A statement from the audience referred to freedom of expression. Parents were said to be responsible for the education and safety of their children and not governments or other stakeholders. It was cautioned not to infringe the right to freedom of expression and not to censor content in the name of protecting children.
Then the question was raised how African countries could get support to deal with issues of child online protection. Amy Crocker stepped in and explained that the foundation is trying to support start up initiatives that would like to establish reporting mechanisms in their countries.
Also the question of self produced sexual content was discussed with regard to the potential criminalisation of young people when they create and distribute such imagery. On behalf of the signatories of the Communiqué Jutta Croll explained that they deliberately did not refer to self produced imagery because to their opinion it should be first and foremost the responsibility of other stakeholders to take care of the protection of children that have not produced those images voluntarily and to ensure that such imagery is removed once it was put in an sexualised context.
Eventually remote speaker Gitte Jakobsen pointed to the investigation perspective of Grey Area images. To their experience they are more useful for identification of victims and prosecution of perpetrators because the offenders seem to be less careful of what

If there were presentations during the workshop session, please provide a 1-paragraph summary for each presentation:

Jutta Croll: Network No Grey Areas on the Internet
In November 2014, the Network against Abuse and Sexual Exploitation of Children – No Grey Areas on the Internet was launched under the patronage of the German Federal Minister for Family Affairs, Senior Citizens, Women and Youth, Manuela Schwesig. The objective of the network is to combat abuse and sexual exploitation of children on the Internet in the so called “Grey Area”. It is built of three pillars: The first pillar is a competence centre at jugenschutz.net that continuously monitors depictions of sexual exploitation of children, analyses the experiences of hotlines and involved services providers, identifies hubs and establishes systematic counter-measures, and cooperates with the involved service providers regarding effective techniques for addressing sexual exploitation of children, including relevant technical indications (e.g. hashes, keywords). The second pillar are service providers, so far Google has joined the network. Service providers investigate reports of content depicting the sexual exploitation of children and take appropriate action. This may involve the removal of such content, reports to authorities where warranted and use of automated mechanisms to attempt to identify and remove re-uploads of known abusive images. They also work with the network to educate and warn users about this type of content by guiding users who come across child sexual exploitation material unintentionally to reporting mechanisms and safe resources and by guiding delinquents to safe resources and help sites. The third pillar are hotlines that receive reports regarding sexual exploitation encountered online, inform the service providers involved in the network as well as the competent authorities (prosecution, foreign hotlines of the INHOPE-association, other service providers) when becoming aware of depictions of sexual exploitation of children on the Internet and contact service providers regarding known cases of sexual exploitation of children, if the competent authorities do not (or can not) act in order to get the material removed. The network is coordinated by the I-KiZ, the German Centre for Child Protection on the Internet. The network deals with so called “Grey Area” imagery i. e. posing and modelling images that may not be illegal in all countries but infringe the dignity, the privacy and the physical integrity of children. Criteria for the classification of images to be “Grey Area” lay in the depiction itself, if the production is harmful for the children depicted or other children might be harmed be confrontation with it, or in the context of the provision of the depiction e.g. the amount of images or sexualized comments by users and potential perpetrators.
Gitte Jakobsen: Drawing the picture – What constitutes the "Grey Area" of (legal) sexual exploitation of children?
Save the Children Denmark have issued the report “Images in the Grey Area” in 2014. They have coined the term “Grey Area” and defined it as follows: ”Images in the grey area are still or moving images of children’s bodies, where the photographer’s staging of the child and / or the use of the material in an erotic or otherwise sexualised context gives grounds to presume that the purpose of the recording or display is to satisfy a sexual interest in children.” The definition is based on the Taylor/Quayle Copine scale, referring to the categories 4 = posing, 5 = erotic posing and 6 = explicit erotic posing.
Save the Children Denmark have analysed a minimum of 16.000 images from around 400 URLs. They found mainly images of girls (85 %), a few boys (9 %) and some with both (6 %), the majority of children were aged 9 – 12. They consider exploitation of the child, when they assume that the child has not given consent to the photographer and no permission to the distribution. Commercial exploitation is presumed when financial profit is made from sexual exploitation of a child, f. e. by the production and sale of child abuse images, including sale or advertising on pornographic and other types of websites with CSAM. Images in the Grey Area are often to be found in a pornographic and commercial context on the Internet.
Marco Pancini: Understanding the concepts – Which concepts of provision and search strategies of perpetrators are in use?
Google is committed to try to fight against any kind of abuse and against this kind of images online. The approach is based on three pillars: Removal of the imagery and reporting the abuser as soon as knowledge is gained about this kind of content, investment in technology to protect from this kind of content, and work with industry and law enforcement in order to keep pace with the criminal activities. Google’s policies are absolutely in line, we don't accept on any of our platforms this kind of content. Google builds also on flagging procedures that allow users to report content they consider inappropriate. In the photo DNA database a digital fingerprint of images that are identified as content to be removed is stored, that is a specific technology to make sure that the same image of the same content cannot go up again. Together with the network partners Google identified more or less 1,000 key words and terms that are related to child abuse material in this area. Every time these key words are put in the search engine, a specific word answer shows up informing the person that they search for something illegal. The person receives a clear notice that the activity of searching and disseminating of the content is illegal. In addition links to support and counselling services in order to help the person in trying to get over his illegal activities are shown. If someone is looking for something specific and this link was removed because it was notified by law enforcement, a removal notification is given to state in the search page that this is a link was removed because the content was illegal. Another powerful tool is to work with our engineers on the search quality to make sure that content related to search on child abuse material – also grey area – is deleted on Google so that it does not come on top of the search results.
Christian Schulze: Analysing the background – How can artificial intelligence and so called multimedia opinion mining help to analyse search terms, phrases and sentences of perpetrators?
The presentation gave an overview on technical tools for classification of content in regard of its inappropriateness and exploitative nature. Such content is searched for by perpetrators using specific vocabulary and abbreviations. To cut across such searches and remove the respective content it is necessary to analyse natural language. From a technical point of view it is difficult to understand opinions, emotions and sentiment by a mere analysis of the words in use. The DFKI uses so called multimedia opinion mining. By this methodology firstly texts provided by a search engine on a certain query are analysed particularly regards terms frequency or the occurrence of certain words in amounts. Then with the analysis results particular classifiers can be trained for detecting search phrases or queries. With that technology it is now not only possible to analyse certain segments of text, but also to take other modalities like images and videos into account.
The sentiment concept analysis uses text classifiers, visual classifiers and acoustic classifiers to support a technology based decision whether someone is searching for a certain type of (illegal) content on the web. Especially with regard to posing images in the Grey Area where the context matters this technology can by of tremendous value.
Grégory Mounier: Addressing the commercial aspects – Which commercial transactions and payment measures are supportive to the dissemination and exchange of CSAI, how do they count for Grey Area material? What role do virtual currencies play? And how can the – real and virtual – cash flow be desiccated?
The European Cyber Crime Centre supports the 280 member states to tackle organized crime and child abuse on line and elsewhere. In terms of child sexual abuse and exploitation we have a team of about 15 specialists working full time on that issue. We are working on the basis of information that is sent by our colleagues from law enforcement in Europe and beyond actually. When it comes to child sexual abuse imagery and commercial exploitation the offenders that are in those types of crimes are extremely IT savvy. We see in that particular community of criminals they are using cutting edge encryption technologies, using all the new materials and technology and methods that can obfuscate online and then they are concerned about anonymity and security. So whenever they have to use payment methods, they will naturally go towards alternative payment methods that means money transfers, digital wallets and cryptic currencies. Payment methods also depend on the type of criminal activity, f. e. with the recent trend of live-streaming of abuse online they're using money transfers because it's just more convenient. Over all there is a massive trend of a shift or migration from traditional payments to alternative payment methods because it's less traceable. And for them it means it's less vulnerable and provides more anonymity. So far the payment for content on commercial websites is 50 % money transfer, 30 percent credit cards and 20 % digital wallets or cryptic currencies.
Akio Kokubu: Reacting to reports - The role of hotlines
In Japan Child Pornography Law Article 3 (iii) was revised in 2014. Considered illegal is “Any pose of a child wholly or partially naked, and explicitly focused on sexual body parts such as genital area, buttocks and chest, and which arouses or stimulates the viewer's sexual desire, i. e. acrobatic posing that explicitly reveals buttocks region in very small bikinis, C-thru clothing that the genital area is visible, purposely showing underwear by flipping the skirt. So, many images considered “Grey Area” are already illegal in Japan. Eleven child welfare organisations among them ECPAT Japan, Think Kids and Lighthouse have pleaded a revision of the Child Welfare Law to ban the distribution and sales of images involving nudity of children aged under 15 years old. They also pleaded for a revision of the Child Abuse Prevention law to address the phenomenon that some children are forced to acts of abuse and exploitation by their parents.
The role of the hotlines is a reacting one, namely they do not search information by themselves to avoid criticism of censorship. User’s understanding for CSAM is necessary to receive reports. Awareness activities to users are very important.
With the information on illegal content received from Internet users, hotlines will generally inform police, submit notice and take down to website administrators and ISPs, inform related Institutions and handle illegal information through international co-operation.
An automatic content rating system on the Internet with image recognition technologies was developed in the past. It was good for a screening tool but confirmation by human eyes was still needed because such technologies did not take into account context of information.
Marie-Laure Lemineur: Syntesising the powers – The Message of the Communiqué "No Grey Areas" adopted by child welfare organisations around the world
Depictions of sexual exploitation of children ("any representation, by whatever means, of a child engaged in real or simulated explicit sexual activities or any representation of the sexual parts of a child for primarily sexual purposes" – so called child pornography) are proscribed by law in many countries worldwide.
However, the sexual exploitation of children is not manifested only through child abuse images. In our modern societies children can be sexualised and treated as sexual objects in many different ways, for example on the Internet through images depicting children in erotic or suggestive poses. These are part of so called “Grey Area” imagery that does not necessarily cross the lines drawn by the law to qualify as child sexual abuse imagery, but is exploitative in nature. It also violates children´s dignity and their right to privacy and damages their sense of security. These images are spread widely throughout the Internet and are much easier to find than child sexual abuse imagery.
Worldwide condemnation and combating those images in the Grey Area of child abuse and exploitation on the Internet is the postulation of organisations world wide, dedicated to the welfare and protection of children. In November more than 30 organisations from many countries around the world have signed a joint Communiqué addressing the condemnation of “Grey Area” imagery. The Communiqué can be found at www.i-kiz.de/NGA2015. The initiative is still open for other child welfare organisations to support and adopt the Communiqué. Please contact jc@i-kiz.de.

Please describe any participant suggestions regarding the way forward/ potential next steps/ key takeaways (3 paragraphs):

Categorisation of content and clear criteria for its assessment were discussed as the important next step in fighting “Grey Area” imagery. There is a need for a worldwide alignment of what constitutes depictions of abuse and sexual exploitation of children on the Internet based on common principles for the assessment of such material. Commonly defined criteria in order to classify such material need to be developed. Terms and definitions must be precise enough to meet the standard of legal certainty; at the same time they must be worded in such a manner that they remain pertinent in dynamic developments.
How can the right to freedom of expression be balanced with the children’s right to protection of their dignity, physical integrity and privacy? The question was brought forward into the main Session on Human Rights on Friday morning and there discussed further. Children have the same rights as any other group in society, but due to the vulnerability there are certain rights given especially to this group. Freedom of expression and protection of children are not in contradiction but there is a need to balance the right of freedom of expression to the right of children's dignity and privacy and the right of children to their physical integrity. Recent research shows that soon to be 1 in 3 Internet users worldwide is under the age of eighteen, and when it comes to developing countries, it is 1 in 2 Internet users. Therefore children should not be considered as a minority but a very important group that's got the same rights.
What is the role of self produced sexual content in the concept of Grey Area Imagery? The question was taken forward to the workshop WS 28 Multistakeholder solutions for youth produced sexual content on Wednesday afternoon. A clear distinction was made between young people producing sexual content in a consenting relationship and children being coerced or enticed in producing images of themselves. The latter need to be protected from being abused for the production of images and from their innocent images being used for sexual arousal of adults and thus being exploited probably also commercially. Adolescents on the contrary argue that producing sexual content has become part of their personal development and is seen by them as a “healthy” kind of sexuality. Nonetheless it is also important for them to become aware that such imagery might be distributed and then misused in a way they would not like.

Estimate the overall number of the participants present at the session:

60

Estimate the overall number of women present at the session:

About half of the participants were women

To what extent did the session discuss gender equality and/or women’s empowerment?

It was mentioned briefly in the presentations and discussions

If the session addressed issues related to gender equality and/or women’s empowerment, please provide a brief summary of the discussion:

The topic was only touched upon briefly in regard to the huge gender differences in the number of Grey Area images assessed by Save the Children Denmark. Of 16.000 images 85 % showed girls, 9 % showed boys and 6 % showed both.

Go back