UI Action Recorder

 

UI Action Recorder :

A Chrome extension that records user interactions with web pages and allows replaying them automatically.

Its a lightweight Chrome extension that enables users to seamlessly record, replay, and export their interactions on any webpage. Designed for automation testers and developers, it captures actions like clicks, inputs, and form submissions with visual feedback, and converts them into executable Selenium scripts in Python or Java. The extension features smart selector generation, visual indicators during playback, customizable playback speed, and an intuitive popup UI. Developed entirely in vanilla JavaScript with Chrome Extension APIs, this tool is ideal for fast prototyping, test script generation, and enhancing UI automation workflows.



🧩 High-Level Flow:

  1. The extension has a popup interface with four main controls:

    • Record: Starts capturing user interactions on the current tab.

    • Play: Replays the last recorded session.

    • Export: Generates Selenium code in either Python or Java.

    • Clear: Wipes all recordings.

  2. Internally, the extension is divided into:

    • popup.js: Handles the UI and interactions.

    • background.js: Manages the state, communication between scripts, and screenshots.

    • content.js: Injected into webpages to capture DOM interactions.

    • manifest.json: Defines permissions and extension metadata.

Features

  • Records user interactions (clicks, inputs, form submissions, etc.)
  • Visualizes elements during recording and playback
  • Captures screenshots of actions
  • Provides a simple interface for recording and replaying
  • Shows visual indicators during recording and playback
  • Adjustable playback speeds (Slow, Normal, Fast, Turbo)
  • Export recordings to Python or Java Selenium scripts

Installation


  1. Clone or download this repository
  2. Open Chrome and navigate to chrome://extensions/
  3. Enable "Developer mode" using the toggle in the top-right corner
  4. Click "Load unpacked" and select the directory containing this extension
  5. The UI Action Recorder extension should now appear in your extensions list

Usage

Recording User Actions

  1. Navigate to the web page you want to record actions on
  2. Click the UI Action Recorder extension icon in your browser toolbar
  3. Click the "Record" button to start recording
  4. Perform the actions you want to record on the web page
    • Elements will be highlighted as you interact with them
    • A red indicator will appear in the top-right corner during recording
  5. Click the "Stop" button in the extension popup when finished

Replaying Recorded Actions

  1. Navigate to the web page where you want to replay actions
  2. Click the UI Action Recorder extension icon
  3. Select a playback speed (Slow, Normal, Fast, or Turbo)
  4. Click the "Play" button to replay the most recent recording
    • A blue indicator will appear in the top-right corner during replay
    • Elements will be highlighted during replay


Managing Recordings

  • The extension popup shows a list of your recordings
  • Each recording includes a snapshot of the actions and a screenshot
  • Click the "Clear" button to remove all recordings


Exporting to Selenium Scripts

  1. Click the "Export" button either in the main toolbar or next to a specific recording
  2. Choose the format (Python or Java)
  3. Review the generated code
  4. Click "Copy to Clipboard" to copy the code
  5. Use the code in your automation projects



Development

The extension consists of the following components:

  • manifest.json: Extension configuration
  • popup.html & popup.js: User interface
  • background.js: Background service worker
  • content.js: Content script that runs on web pages



Limitations

  • Complex interactions with shadow DOM elements may not be captured correctly
  • Some dynamic websites might change their structure between recording and playback
  • Elements that change or disappear might cause replay failures
  • Generated Selenium code may need adjustments for complex scenarios

Comments

Post a Comment

Popular posts from this blog

Heleanium - Web without using Docker

Image
 Heleanium - Web without using Docker Official Heleanium Website: https://healenium.io/docs/overview Youtube Video Link : https://www.youtube.com/watch?v=EF2gM16TnJg 1. We need install first :  Java 11+ , Python ( 3.12.0 )&  PostgreSQL ( prefer to use latest 16 version ) 2. Once PostgersSQL installed ( make sure during installation whatever you put the password for the same you remember / noted down, need this is in 3rd step), open SQL Shell ( psql ) from the applications, as shown below: 3. Once you open the Shell , hit ENTER till Username & provide same Password as in the step 2.  4. Now , feed one by one on the above terminal, this will create the DB ,  User & Schema for you: 4.1 : CREATE DATABASE healenium;      4.2 : CREATE USER healenium_user WITH ENCRYPTED PASSWORD 'YDk2nmNs4s9aCP6K' ;      4.3 : GRANT ALL PRIVILEGES ON DATABASE healenium TO healenium_user;      4.4 : A...
Read more

The self healing automation framework - Healenium Updated Dec2023

Image
Healenium  Architecture   Healenium  Official Github Link :        https://github.com/healenium And this complete project E2E available at my repo as well      https://github.com/madanankurmalviya/HealeniumTestOnJavaSeMvnProject.git Youtube demo for complete setup :       https://www.youtube.com/watch?v=Xg_sqE4VUyU 1. Healenium - Web from Maven Reporsitory: <dependency>     <groupId>com.epam.healenium</groupId>     <artifactId>healenium-web</artifactId>     <version>3.4.8</version> </dependency> 2. Check for Selenium-Java latest dependencies: <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java --> <dependency>     <groupId>org.seleniumhq.selenium</groupId>     <artifactId>selenium-java</artifactId>     <version>4.16.1</version> </depend...
Read more

Concept of creating the Self Healing Model - ( InProgress )

Image
Hey folks, I'm currently working on implementing the concept of self-healing. Below is the initial draft of my script. I'll keep you posted on my progress and provide updates once I complete this task : Step 1: Utilize Selenium to scrape the current page and save the elements in a CSV file as outlined below. Step 2: Capture the attributes of each element and store them in a database. This can be done either manually or by implementing a script to update the scraped elements into the database. Step 3: Implement Self-Healing mechanisms for situations where the attribute of an element undergoes changes. Tests should be capable of identifying the correct element based on its alternative attributes, employing Machine Learning (ML) techniques. a. Train the ML model to determine the probability of each element. b. Training requires two datasets in CSV format. Step 4: Initiate model training exclusively when element-not-found exceptions (NoSuchElementException) occur. This ensures that...
Read more