Advise for Bachelor students - by Michael Sedlmair last edited: Oct 13, 2014 I put together (1) a typical structure of how Vis/HCI theses look like in our VDA group, (2) some recommendations on how to conduct and report your research, (3) specifics on the process and deadlines, and (4) a list of mandatory and recommended readings. Specifically (1) and (2) are not set in stone, but are rather meant as a loose set of guidelines to help you prepare, conduct, and write your BSc thesis. This document is written from my own personal perspective, however, others in the VDA group (e.g., Torsten Moeller, Tom Torsney-Weir, ...) follow the same (or at least very similar) guidelines. If your thesis is supervised by one of them and you have specific questions regarding this document, please do not hesitate and ask them for clarification! ************************************************************************ (1) Suggested structure for your thesis ************************************************************************ Most of the theses I supervise can be classified as visualization technique, design study, or evaluation papers: http://ieeevis.org/year/2014/info/call-participation/paper-submission- guidelines (see 5. paper types) The following outline should work well for many *technique* and *design study* theses (most of the theses). 1. Motivation * motivate your topic * clearly say what the problem is and why other related techniques/tools don't solve it * clearly state the goals of your work; discuss the metrics how you can measure success (e.g. my new algorithm will be faster, people will be able to perform a certain task more quickly and with fewer errors, better user satisfaction, people will gain new insights, I can do something that has not been possible before but is important, ...) Note: you will have to show later that you actually met the goals you claimed here! * briefly explain the process of your work and the methods you used (e.g. collaborating with domain experts, multiple prototypes, ... ) * crisply summarize your main contributions at the end of the motivation section 2. Related Work * are there similar ideas to your idea? (Note: most likely, yes!) * Find literature ** scientifc: see below ** non-scientific: browse the web and report general ideas that you find related to yours * for all related work: ** briefly explain what it is about and why it is related ** reference it properly ** say how your work is different and goes beyond it/is better 3. Design * Conceptually describe what you have done * Explain how it works and why it is cool (i.e. helps solving the problem) * screenshots! * Justify design decision: ** explain alternatives that you thought of ** support your arguments either by literature that supports your decision ** or by thoroughly justifying it based on the problem at hand * Note: Don't give technical implementation details here, this comes in the next section 4. Implementation * briefly explain how you engineered your tool/technique * this section can be short! * focus on the interesting bits here, you do not have to explain all your code 5. Evaluation and Discussion * how did you evaluate that you reached your goals? * did you use specific methods to do so? (e.g., a user study, a case study, usage scenario, formal mathematical proof, performance tests, (visual) comparison to state of the art, ...) * explain methods and results of this evaluation * discuss the results: here you should refer back to the most closely related work. Now you can say why your new thing is better (given the metrics outlined in the motivation). * discuss lessons learned and unexpected findings * Note that in a bachelor thesis you will often use "simple" validation methods, such as usage scenarios with example datasets, plus a solid discussion of the benefits. Note, however, that an evaluation -- even a small one -- is mandatory for a scientific thesis. This section, therefore, must not be missing. 6. Conclusions and Future Work * give a concise summary of your main findings/insights/contributions * outline future work ** future work can be concrete next steps in this particular project ** future work can be also more broadly such as streams of research that other people should focus more on 7. References * provide a full list of references * all references need to be cited in the text Appendix / Supplemental Material (if any) Evaluation / User study papers is another common form of theses we offer. Their layout differs slightly from the template I provided above. For user study papers, please follow the outline provide by Saul Greenberg: http://pages.cpsc.ucalgary.ca/~saul/hci_topics/assignments/ controlled_expt/ass1_reports.html Note: Many of the more general aspects of the template above might still be helpful (e.g. Motivation, Related Work, ...). It might therefore still be helpful to carefully read and think it through. *** Bean-counting *** I often get asked how many questions, e.g. how many pages, references, etc. a thesis needs. Here are some guidelines of what we in the VDA group usually expect: * pages: 8-10 (using the template below) * if you have any spacious material, such as long code snippets, large and/or many screenshots, that you want to include but that are not core relevant, you can include them as supplemental materials * references: As a rule of thumb, we are expecting 20+ references. At least 15 of those should be from scientific literature, that is research papers or scientific books. Important note: We are not at all grading your thesis by bean counting! Rather we look at it in a more holistic way, judge the contributions, ideas, design, implementation, evaluation, discussion and presentation. That said, if you for instance "only" have 5 pages, but these really rock and have everything relevant in it, you of course still would get a good grade. Note, however, the challenge usually is the other way round: Instead of filling pages you might find it challenging to fit everything into the page limit. *** Template *** Please use the template for IEEE VIS conference papers: http://www.cs.sfu.ca/~vis/Tasks/camera.html *** Submission *** Please submit the following via email (before the deadline, see below): * Pdf file of the thesis * (if applicable) running prototype (for instance, on the web) If you have developed software this is mandatory. Note, I do not have the time to run complicated instructions to get your prototype to run. That is, preferably your tool should run without any extra software, etc. However, I do know that in some cases that might not be feasible. If so, please clarify it with me upfront (i.e. before you send me the first version of your prototype). In this case, include a readme.txt in which all steps are explained properly. * (if applicable) documented code If you have developed software this is mandatory. * (if applicable) additional screenshots or a video * (if applicable) study protocols and data ************************************************************************ (2) Other recommendations on how to conduct and report your research ************************************************************************ *** Finding Scientific Literature *** Most of the topics we are supervising are in the areas of Visualization, Data Analysis, HCI, and Computer Graphics. Here are some good starting points to retrieve interesting literature: Papers from visualization conferences: * IEEE InfoVis, * IEEE VAST, * EuroVis, * TVCG HCI, also some Visualization: * ACM CHI Computer Graphics (which are usually not supervised by me): * Siggraph + Siggraph Asia * Eurographics * ToG The papers are accessible via: * IEEE Xplore: http://ieeexplore.ieee.org/Xplore/home.jsp * ACM digital library: http://dl.acm.org/ (you need to be on campus or logged in via vpn to access them) Alternatively you might find them via: * google scholar * or simply google Note 1: These conferences will most likely give you good starting points. However, it does not mean that you should not consider other Journals and Conferences, specifically if you have a topic that goes beyond generic visualization and HCI. Note 2: It is a good idea to engage in "snowball" exploration, i.e. following up on interesting references that you find in core-relvant papers. Note 3: Do your literature search before starting the project and take notes. Otherwise, you will have forgotten your thought when it comes to writing up your thesis. Note 4 (important): While most of the literature will go into the related work section of your thesis, also other sections of your thesis should be backed up with literature, e.g.: * Motivation: Who dealt with similar problems, e.g. name the most important ones and briefly explain them (and then explain them in more detail in the RW section) * Design: Cite papers that justify your design decisions (e.g. because they have conducted a comparative study, or took a similar decision that turned out to work well) * Implementation: Cite frameworks, etc. that you used, e.g. D3, pbrt, paraview, etc. * Eval/Discussion: Revisit related work and compare and discuss them given the new insights you have gained through your work. *** Writing your thesis *** * Everybody should take a look at how scientific writing is done properly: http://engineering.missouri.edu/civil/files/science-of-writing.pdf * Writing up your thesis is the final step of your BSc project, either after you have done all the practical work, or at least a huge portion of it. * Your thesis can be in English or German. I prefer English, as almost all of the relevant literature is in English and as it gives you the opportunity to work on your English writing skills. English language and grammar will *not* be graded (unless your thesis is totally incomprehensible, which however can similarly happen in German) * Tamara Munzner has some helpful advises on writing up visualization research, which I would highly recommend. I specifically repeat some of those here: * "I highly recommend that you write an outline before you start writing prose. If you have a tendency to get stuck in wordsmithing, consider a sweepline approach, where you do multiple outline passes to fill in levels of detail. Start with section titles. For your next pass, fill in subsection titles. For your next pass, have one bullet point per paragraph. For your final pass, have a few bullet points per paragraph of the main points you want to make. You monotonically advance through the paper, you don't get to go back and fiddle with what came before until the next pass. Then, and only then, write up in words." from: http://www.cs.ubc.ca/~tmm/policy.txt * "Writing up is typically a combination of bottom-up (work up from results) and top-down (work down from master plan). Consider starting writing the paper shell (intro, previous work, algorithm) even before you have final results. A more extreme approach that's worth trying is to do this before you've started programming *anything*. Then we can decide if it sounds like a strong enough project to bother doing it! It's also a lot easier for us to discuss the algorithm you have in mind if you've written it down, and we may be able to improve it before you spend a long time on coding. It's almost always true that once you have the results, you will drastically change the spin of those early sections. But you'll probably end up with a better paper if you do it this way, and maybe even spend less total time." from: http://www.cs.ubc.ca/~tmm/policy.txt * At a lower level, see writing style discussion from Tamara: http://www.cs.ubc.ca/~tmm/writing.txt Following these guidelines, will also help to improve your English writing skills. *** Common Pitfalls *** Many interesting and recurring pitfalls in visualization research can be found in this paper: Munzner, 2008: Process and Pitfalls in Writing Information Visualization Research Papers http://www.cs.ubc.ca/labs/imager/tr/2008/pitfalls/pitfalls.pdf Read it! It will make your life easier! *** Misc *** * Don't forget to backup all your work! * IMPORTANT: it is a good idea to develop some passion for your work your work will be better, your thesis will be better, your talk will be better ... i.e. it is important to pick a topic that you like to work on, even though it might seem to be more work than another one! ************************************************************************ (3) Specifics on the process and deadlines. ************************************************************************ *** Work load *** A BSc thesis at the faculty of computer science at Uni Wien is worth 18 ECTS. One ECTS is equivalent 25 hours of work per semester. Hence, assuming 18 weeks for one semester (given our deadlines), we are expecting you are putting in around 25 hours of work every week for your BSc work. This is the basis of our grading. *** Meetings *** You will have 3 meetings with your supervisor (i.e. me if I turn out to be supervising your work), each of them approx. 30-45 min, and a final presentation. 1. meeting (~after 2 weeks) The first meeting should be in the beginning *after* you: * thought through the problem, * did the first round of literature analysis, * had some first ideas for solving the problem, * have generated a time table with adequate milestones, and * collected questions that you would like to discuss with me. Good preparation of these steps (e.g. slides) will significantly help to make the meeting effective for you. During the meeting, we will focus on the following items: * fix milestones * concrete "thesis proposal" with prototypes milestones, etc. (the "thesis proposal" will then act as a sort of "research contract" between us: in written form, it clearly specifics the goals that you want to have reached at the end of the thesis.) 2. meeting (after 2 month, halftime) In the second meeting we will talk about the design/solution that you propose. For that, you will need to have done: * iterative and parallel prototyping * started coding the solution and have a first running software prototype During the meeting, we also will: * check on milestone * status update * discuss issues and suggestions on how to go around them 3. meeting (2 weeks before final presentation) In the third meeting, we will discuss the fully-implemented technique/tool you build, derive final todos wrt the technique/tool, and jointly decide on how to evaluate it. To do so, please sent me at least 24h before the meeting: * your technique/tool, or a link to it * if necessary: a quick howto about how I will get it running, and how to use it (susually not more than 10-20 lines/bullet points) * If your technique/tool needs additional software to be install, please let me know at least one week before the meeting. During the meeting, we also will: * do another status update * check milestones and advancement * discuss issues and suggestions on how to go around them For all meetings: * It is your responsibility to set a meeting time with me. The best way of getting in touch with me is email. * Be prepared for the meetings! Think about the points you want to talk about (agenda), prepare software demos (if you have any)! Send me any material you want to look at before the meeting at least 24h before the meeting. * Plan ahead: That is, do *not* have your first meeting 2 weeks before the submission deadline! * This is just a standard way of dealing with meetings. As I have many Bachelor students, I cannot do weekly meetings with all of you, although I would like to. However, based on your topic, motivation, and passion for your thesis we might end up meeting more frequently. 4. Final presentation After you are done (or when you are close to the end), you have to have a presentation in front of a broader audience. You will be given 15 minutes to present your work + 10 minutes of discussion (usually). * Prepare slides, practice your talk. * The final presentations are usually in the last 2 weeks of the semester in which you are attending the Bachelor seminar (that is you also should not try to cram all the work into the very last month! * The presentation should be preferably in English Talk guidelines: * Your presentation should include a live demo of the tool you have build (the demo is part of the 10 minutes) * focus on the most interesting parts! You don't have to talk about everything you have done, all the details will be in the thesis they don't have to be in the talk! * Some details are necessary: e.g., #users in a study * you also do *not* have to exactly follow the outline of your written thesis! Often, it is not even necessary to talk about every section! * Practice the talk!!!! Stop the time when practicing: 10 min is not a lot of time! Here are two models that I personally find helpful when preparing (good) talks * Onion model: rather than telling a sequential story with a surprising end, you should think of the talk as rings of an onion. You start with the very core including the problem, goals but also the most important findings and contributions. Then step by step you add more details to this core, the rings of the onion. It, for instance, might be a good idea to start with the demo so people know what it is all about. Your audience will not have the same background on your topic neither will they have though so deeply as you about it! * Prototyping T model: Give you audience a quick overview of the breadth of what you did (the horizontal line in the T), but only go into depth for some parts of it (the most interesting ones, the vertical line in the T, note: you can go into depth with more than one thing of course, but be cautious not trying to fit too much things into your talk) *** Deadline for submission *** The deadline for your final submission is defined by when you take the seminar: "Praktikum mit Bachelorarbeit". If you do so in the summer term (SS), the final deadline is: * August 1 after the semester you took the seminar If winter term (WS), the final deadline is * March 1 after the semester you took the seminar That is, you get one extra month after the semester to finish things up. I am happy to give you a quick feedback on a first draft of your thesis. In this case, I need the draft no later than 1 month before the final deadline. Due to a chronic lack of time, I can only give max one round of feedback. *** Grading *** Your work will be evaluated according to the following grading scheme: First meeting: 10% of the grade * quality of milestones / lit search ... Second meeting: 10% * quality of prototypes * quality / quantity of progress Third meeting: 10% * quality of final tool * quality / quantity of progress Presentation: 10% Thesis: 60% * 40%: Content (quality of proposed solution, conducted evaluation, etc.) * 20%: Written report 100-88%: grade 1 87-75: grade 2 74-63: grade 3 62-50: grade 4 < 5: grade 5 (failed) ************************************************************************ (4) Readings (Vis thesis) ************************************************************************ The following papers and books are highly recommended to read, if you not already have done so in your HCI / Vis class. Mandatory (M) / everything else is recommended but not mandatory per se. *** General *** * (M) Munzner, 2014: Visualization Analysis and Design http://www.cs.ubc.ca/~tmm/courses/533/book/ --> fundamental book on visualization * (M) Munzner, 2008: Process and Pitfalls in Writing Information Visualization Research Papers http://www.cs.ubc.ca/labs/imager/tr/2008/pitfalls/pitfalls.pdf --> describes common pitfalls in writing visualization papers and theses *** For design study papers *** * (M) Sedlmair et al., 2012: Design Study Methodology: Reflections from the Trenches and the Stacks http://homepage.univie.ac.at/michael.sedlmair/papers/sedlmair2012infovis -1.pdf --> This paper describes the underlying methodological approach for design studies. If you are conducting a design study, I recommend to use it as your methodological framework. Examples of good design studies: * Weaver 2007: Visual Exploration and Analysis of Historic Hotel Visits. http://www.cs.ou.edu/~weaver/academic/publications/weaver-2007b.pdf * Van Wijk, 1999: Cluster and Calendar based Visualization of Time Series Data. http://www.win.tue.nl/~vanwijk/clv.pdf * Sedlmair et al., 2011: Cardiogram: Visual Analytics for Automotive Engineers http://homepage.univie.ac.at/michael.sedlmair/papers/sedlmair2011chi.pdf * Baur et al., 2010: The Streams of Our Lives: Visualizing Listening Histories in Context http://homepage.univie.ac.at/michael.sedlmair/papers/baur2010infovis.pdf * Boosherian et al, 2012: Facilitating Analysis of Trade-Offs, Uncertainty, and Sensitivity In Fisheries Management Decision Making http://vismon.org/Vismonpaper-EuroVis12.pdf *** For user study papers *** (will come soon)