Calls | TIME 2027

Important Dates

Workshop Paper Submission DeadlineJanuary 4, 2027 (AoE)
Workshop Paper NotificationFebruary 1, 2027 (AoE)
Workshop Paper Camera-Ready DeadlineFebruary 16, 2027 (AoE)
The Web Conference 2027May 10–14, 2027, Dublin, Ireland

All deadlines are in the Anywhere on Earth (AoE) time zone. The submission portal is now open on OpenReview.

Submit on OpenReview Download CFP (PDF) ACM LaTeX Template

Call for Papers

The central question: How should we evaluate Web AI systems when capability alone is not enough? TIME 2027 invites work that moves beyond headline scores to examine how modern AI systems perform, adapt, fail, and affect people in realistic settings.

Foundation models, generative and multimodal AI, retrieval-augmented generation (RAG), and AI agents are rapidly reshaping search, recommendation, knowledge access, and other Web applications. Yet their performance cannot be captured by task accuracy alone. Systems may produce convincing but incorrect answers, fail when information changes, misuse tools, behave inconsistently across users or contexts, or introduce safety, privacy, and fairness risks.

The 3rd International Workshop on Transformative Insights in Multi-faceted Evaluation (TIME 2027) brings together researchers and practitioners to advance rigorous, responsible, and actionable evaluation of these emerging Web AI systems. Building on the TIME 2025 and TIME 2026 workshops, this edition places particular emphasis on foundation models, multimodal and generative systems, RAG, and agentic Web applications.

We seek to move evaluation beyond static benchmarks and single headline metrics toward evidence about how systems behave in realistic, interactive, and changing environments. We welcome work that examines not only what AI systems can do, but also how reliably, robustly, safely, fairly, transparently, and efficiently they do it, and whether their outcomes deliver meaningful real-world utility. Contributions that offer reproducible methods, reveal important failure modes, or translate evaluation evidence into practical recommendations are especially encouraged.

TIME 2027 welcomes perspectives from Web research, information retrieval, machine learning, multimodal learning, responsible AI, security, privacy, and human-centred AI. Our goal is to build an interdisciplinary community around a shared challenge: making the evaluation of modern Web AI more meaningful, comprehensive, and useful in practice.

Topics of Interest

Topics include, but are not limited to, the following areas:

Area	Topics
1. Foundation Models and Generative AI	Capability assessment across tasks, domains, and user needs; factuality, hallucination, and evidence-grounded generation. Robustness, generalisation, failure analysis, and evaluation under distribution shift. Multi-dimensional assessment beyond aggregate scores and conventional task metrics.
2. Multimodal AI and Web Systems	Evaluation of systems that integrate text, images, video, audio, and other modalities. Cross-modal consistency, alignment, reasoning, and generation quality. Performance and reliability under heterogeneous inputs and changing Web content.
3. Retrieval-Augmented Systems and AI Agents	RAG evaluation, information grounding, retrieval quality, and source attribution. Agent planning, tool use, multi-step execution, error recovery, and long-horizon tasks. Behaviour in interactive Web environments with incomplete, conflicting, or evolving information.
4. Trustworthy, Responsible, and Human-Centred Evaluation	Safety, security, privacy, fairness, bias, and societal impact. Interpretability, transparency, uncertainty quantification, and calibration. Human-centred evaluation, usability, real-world utility, and cross-cultural or cross-domain differences.
5. Benchmarks, Metrics, and Reproducible Methodologies	Evaluation frameworks, metrics, protocols, test suites, and benchmark design. Dataset quality, annotation reliability, validity, reproducibility, and statistical rigour. Open evaluation tools, transparent reporting, and cross-domain assessment.
6. Efficiency, Deployment, and Emerging Challenges	Computational cost, latency, scalability, and sustainability. Evaluation, auditing, and monitoring in real-world deployment. Trade-offs among capability, reliability, safety, cost, and user utility; industry lessons, negative results, and overlooked failure modes.

What Contributions Are We Looking For?

We encourage a broad range of submissions that deepen understanding of AI evaluation, including:

Research papers presenting novel evaluation methods, empirical studies, or systematic investigations.
Surveys and systematic reviews synthesising progress, exposing gaps, and identifying future directions.
Benchmark and metric analyses examining the strengths, limitations, and validity of current evaluation practices.
Evaluation frameworks and tools enabling practical, transparent, and reproducible assessment.
Negative results and failure analyses revealing weaknesses, risks, and failure modes that conventional evaluation may overlook.
Position papers and industry case studies proposing research priorities or sharing lessons from real-world use and deployment.

We particularly value contributions that go beyond reporting a performance gain to explain what an evaluation reveals, why it matters, and how it can inform better research or deployment decisions. Papers introducing a model or application are welcome when they include substantial, systematic evaluation relevant to the workshop.

Submission Requirements

TIME 2027 will use OpenReview for paper submissions and the review process. The submission portal is now open. All papers must be written in English and use the ACM conference double-column template. Submissions will be evaluated through a double-blind peer-review process; authors must anonymise their papers and follow the detailed instructions provided on the submission portal.

Long papers: up to 8 pages of paper content, plus references and an optional appendix, with a maximum total length of 12 pages.
Short papers: up to 4 pages in total, including references and any appendix.
Review: double-blind expert peer review; see Review Process and Publication for the review stages.
Publication: accepted papers will be included in the WWW 2027 Companion Proceedings, subject to compliance with the camera-ready deadline and applicable ACM publication requirements.

All submissions should present original work, comply with relevant research-ethics requirements and applicable ACM policies, and clearly explain the evaluation question, methodology, findings, and significance to the TIME 2027 scope. Papers will be assessed for relevance, technical quality, clarity, methodological rigour, and the value of the evaluation insights they offer. Accepted contributions may be presented orally or as posters, as appropriate.

Review Process and Publication

Building on the process used for TIME 2025 and TIME 2026, TIME 2027 will use a multi-stage, double-blind review process designed to support rigorous, constructive, and fair evaluation.

Initial screening. The Program Chairs will check submissions for completeness and compliance with the required format and page limits. Submissions passing this screening will undergo automated checks for AI-generated content and potential reference hallucinations.
Expert review and author response. Submissions passing the initial checks will be reviewed by at least three expert reviewers, including at least one reviewer from industry. Preliminary review results will be shared with authors, including reviewer comments, AI-checking reports, and Program Chair feedback on formatting. Authors will have an opportunity to respond to the feedback and revise their submissions.
Area Chair assessment and final decisions. Area Chairs will assess revised submissions by considering reviewer comments and ratings, the paper’s strengths and contributions, its relevance to the workshop, and the authors’ responses. The Program Chairs will make the final decisions based on the reviews, Area Chair recommendations, overall paper quality, and fit with TIME 2027.

Paper awards. High-quality papers recommended by the Area Chairs will be considered by the Award Committee for the workshop’s Best Paper and other paper awards.

Publication. Accepted papers will be included in the WWW 2027 Companion Proceedings, subject to meeting the camera-ready timeline and applicable ACM publication requirements. For examples of previous editions, see the ACM Digital Library collections for TIME 2025 and TIME 2026.

Author Guidelines

Formatting Requirements: Submissions must be in English, in double-column format, and must adhere to the ACM template and format (also available in Overleaf). Word users may use the Word Interim Template, and the recommended setting for LaTeX is:

\documentclass[sigconf, anonymous, review]{acmart}

Submissions must be a single PDF file:

Long papers: 8 (eight) pages for the main paper, with additional pages for references and an optional appendix (e.g. details on reproducibility, proofs, pseudo-code), up to a maximum of 12 (twelve) pages in total. The first 8 pages should be self-contained, since reviewers are not required to read past them.
Short papers: 4 (four) pages, including references and the optional appendix.

Double-Blind Review Policy: Paper submissions must conform to the double-blind review policy, meaning that authors should omit their names and affiliations from the paper. All papers will be peer-reviewed by experts in the field. Acceptance will be based on relevance to the workshop, scientific novelty, and technical quality.

AI-Generated Content Disclosure: Authors are required to disclose any use of tools such as ChatGPT or other AI-based systems in the creation of the paper, including text generation, analysis, or other content. If AI tools are used, authors must provide proper attribution and use the tools ethically. AI tools should not be listed as authors; only human contributors should be credited as authors.

OpenReview Profile Completion: Anyone who plans to submit a paper as an author or co-author must create or update their OpenReview profile by the paper submission deadline. Complete profiles are required for assignment and conflict detection. Papers with authors who have not completed their profiles may be desk-rejected.

Exceptions and Accommodations: Deadlines and policies will be strictly enforced. There will be no exceptions or accommodations for late submissions, misconfigured author or co-author accounts on OpenReview, or changes to author lists after the paper submission deadline.