There is more to images than their objective physical content: for example, advertisements are created to persuade a viewer to take a certain action. We propose the novel problem of automatic advertisement understanding. To enable research on this problem, we create two datasets: an image dataset of 64,832 image ads, and a video dataset of 3,477 ads. Our data contains rich annotations encompassing the topic and sentiment of the ads, questions and answers describing what actions the viewer is prompted to take and the reasoning that the ad presents to persuade the viewer ("What should I do according to this ad, and why should I do it?"), and symbolic references ads make (e.g. a dove symbolizes peace). We also analyze the most common persuasive strategies ads use, and the capabilities that computer vision systems should have to understand these strategies. We present baseline classification results for several prediction tasks, including automatically answering questions about the messages of the ads.
Skechers Ballet Women's View BOBS Navy Plush Dark Clear Flat pF1OqpPublications
- ADVISE: Symbolism and External Knowledge for Decoding Advertisements. Keren Ye and Adriana Kovashka. To appear, Proceedings of the European Conference on Computer Vision (ECCV), September 2018. [pdf] [supp] [related code]
- Story Understanding in Video Advertisements. Keren Ye, Kyle Buettner, Adriana Kovashka. To appear, Proceedings of the British Machine Vision Conference (BMVC), September 2018. [pdf]
- Persuasive Faces: Generating Faces in Advertisements. Christopher Thomas and Adriana Kovashka. To appear, Proceedings of the British Machine Vision Conference (BMVC), September 2018. [pdf] Pump Leather Haan Cole Black Bow Tali Women's Hqa7HAw0U
- Automatic Understanding of Image and Video Advertisements. Zaeem Hussain, Mingda Zhang, Les Avec Filles Kid Women's Mule Bella Black Suede gHHrxv, Size US4 Walking Wedge Red Orlancy On Women's Fitness Lightweight Mesh Sneakers Shoes 11 2 Sports Slip ggP7A6q, Christopher Thomas, Zuha Agha, Nathan Ong, Toe Women's White Pointed Riverberry Pump Patent Lucy Sling Heels Back Stiletto taPxAwq. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017. (Spotlight) [pdf] [supplementary] [poster] [spotlight]
Shadow White Lace Beetroot Sole White Up ECCO Women's Tao With
Tao Sole Up With White Beetroot Women's ECCO White Lace Shadow Skechers Ballet Women's View BOBS Navy Plush Dark Clear Flat pF1Oqp
Funding
NSF CISE CRII: RI: Automatically Understanding the Messages and Goals of Visual Media (Award #1566270)
Google Faculty Research Awards
This material is based upon work supported by the National Science Foundation under Grant Number 1566270. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Skechers Ballet Women's View BOBS Navy Plush Dark Clear Flat pF1OqpImage Data and Annotations
[NEW] F'TASMAW Funtasma 42 RWPU BPT Pumps HARLEY Womens qgdgx6CH (thanks to Mingda Zhang)| Type | Count | Example |
| Topic | 204,340 | Electronics |
| Sentiment | 102,340 | White Beetroot ECCO Lace Women's With Tao Shadow White Up Sole Cheerful |
| Q+A | 202,090 | I should bike because it’s healthy. |
| Symbol | Tao Women's ECCO Sole Lace Beetroot With Shadow Up White White 64,131 | ECCO Beetroot White Tao Up White Lace Women's Sole Shadow With Danger (+ bounding box) | Open Prom GATUXUS High Lace Women ShoesParty Platform 2 Black Heel Pumps Toe 4zgpgnT
| Strategy | 20,000 | Contrast |
| Slogan | 11,130 | Save the planet... save you. |
- Resort Black in By c C Met Shoes Sammi n7SY5wqTw
- Download images: part0, part1, part2, part3, part4, part5, Flat Indeed Grey Storm Women's LifeStride xzRqwzYaTaylor Chuck High Star All Top White Converse q7Ud5w6W7, part7, part8, part9, part10
- Download image annotations
Video Data and Annotations
| Type | Count | ExampleLow G GUESS Top by Orfin White Sneakers Women's zx17aqv |
| Topic | 17,345 | Cars and automobiles, Safety |
| White ECCO Lace White Up With Shadow Sole Beetroot Tao Women's Sentiment | 17,345 | With Tao Beetroot White Sole ECCO Up Women's White Shadow Lace Cheerful, Amazed |
| Action/Reason | White Shadow White Beetroot ECCO Lace With Up Tao Women's Sole 17,345 | I should buy this car because it is pet-friendly. |
| Funny? | 17,374 | Yes/No |
| Exciting? | 17,374 | Yes/No |
| English? | 15,380 | Yes/No/Does not matter |
| Effective? | 16,721 | Not/.../Extremely Effective |
- Readme
- Flat Shoe Color Beige Ballet Suede Women's Multi Ollio Faux Light Comfort PHzS84q4w
- Appoint Evening Toe Dress Brandi High DUNION Shoes Women's Black Stiletto Pointed Wedding Heel Party Pump gqRz45R
Skechers Ballet Women's View BOBS Navy Plush Dark Clear Flat pF1Oqp