Visym Collector - Frequently Asked Questions (FAQ)

Frequently Asked Questions (FAQ)

What is Visym Collector?

Visym Collector is a mobile app designed for consented and privacy preserving dataset collection for AI applications.
We pay collectors to use our mobile app to submit "collections", which are videos with labeled objects, activities and attributes used for training visual AI.

How do I become a paid collector?

We are regularly running collection campaigns. Send us an email at info@visym.com expressing your interest in joining the collection team and we will reach out with next steps when a new collection is available in your area.

How do I get started as a collector?

Read the quickstart document for new collectors for instructions for downloading and using the mobile app.
Join the community for discussion with other collectors and annoucements by the collector team.
Look out for periodic emails from the collector team (collector@visym.com) with your dashboard containing ratings and payments for your videos.
Keep up the good work!

I can't log-in!

Check that there are no extra spaces at the beginning or end of your email address
Check that there are no typos (e.g. gmail.con) and that you are using the email address you used to register without any additional capitalization.

What happens to the videos after I submit them?

Collector is a platform to collect and curate video datasets for visual AI. When you submit a video to us, we prepare it for use by the research community for training a visual AI system, in the form of videos available for public download.
An example of a publicly released dataset containing videos provided by the worldwide collection team is Consented Activities of People.

How do I keep track of the videos I have submitted?

When a collector is onboarded you will be sent a welcome email that contains a link to your collector dashboard. This dashboard contains all of the videos that you have submitted to us, along with the ratings from the review team. You can use this to track your expected payments for the week.
Your dashboard is also available by logging using the same email and password that you use in-app at https://collector.visym.com
Please note that your dashboard updates approximately every hour, so there will be a lag between when you collect and when you see this updated. Submitted videos are available for review immediately in-app. Reviews for your videos are usually available within 24 hours.

How do I use the editor to correct a mistake during collection?

We rely on the collector community to submit us videos with accurate annotations in the form of bounding boxes for objects and start and end times for activities. If you make a mistake in annotation, you can fix it after you record using the video editor.
Read the video editor tutorial document for new collectors for instructions on how to use the video editor to correct object boxes and activity times.
Watch the tutorial video from the collector team. Please enable closed captions to improve your experience.

Why did the collections available in the app change?

We are regularly updating the collections and projects available in the mobile application. We may enable or disable collections on demand, based on the needs of our collection, and you may see this as collections dissappearing from your app, and replaced with new collections. This is expected.
We usually update the collections around 8am ET daily.

What do the "Ratings" mean in my collector dashboard?

Each video that you submit is looked at by our review team and assigned one or more ratings according to our review criteria. The activity ratings are a short summary of the output of this review procedure, where the descriptions of the ratings are provided below. If your submission was not accepted, review these reasons to correct your submission next time.
Good. This activity was rated by the review team and passed our review criteria.
Perfect. This submission passed our review criteria, and is an excellent example of this collection which can be used to train other collectors. Great job!
Rate limit exceeded. Each collection can be submitted as many times as you choose, however we will use for payment only the last video submitted for each collection type within a specified time window. We do this to enforce diversity so that we get videos that are separated across days. For example, if the rate limit is 12 hours, you may submit a video for collection "Ride a Bicycle" at 6pm, then again at 6am the next day.
Invalid activity count (M<N). Each collection has a specified number of activity buttons. We require that you press each activity button at least once. Please review your videos carefully before submission, and retake them if there are an incorrect number of activities present. The review (M<N) refers to the fact that your video contained only M activities, but we expected at least N.
Incorrect timing. The start time or end time of the labeled activity is not precise enough. Please review your videos so that the activity starts just after the button is pressed, and the activity ends just before the button is pressed again. We require that at least 50% of the activity is present within this temporal window. If we cannot tell that the activity occurred within the quicklook summary, then it will be flagged as incorrect timing. You may correct this timing after submission or after receiving your review by using the in-app editor.
Box too big. The bounding box surrounding the actor performing the activity is too big. We require that the actor is completely inside and covers at least 50% of the box. The ideal box is one that is tight around the actor both in width and height. You can change the box by dragging the corners in the collection UI. You may also correct the box after submission or after receiving your review by using the in-app editor.
Box too small. The bounding box surrounding the actor performing the activity is too small -- the actor is partially outside the box. We would prefer if you err on a box slightly too big with the actor completely inside. The ideal box is one that is tight around the actor both in width and height. You can change the box by dragging the corners in the collection UI. You may also correct the box after submission or after receiving your review by using the in-app editor.
Box not centered. The bounding box surrounding the actor performing the activity is not properly centered on the actor. We require that the actor is completely inside and covers at least 50% of the box. You can change the box by dragging the corners in the collection UI. You may also correct the box after submission or after receiving your review by using the in-app editor.
Incorrect label. The activity performed in the video does not satisfy the collection definition. Please read the definition carefully and try again. This most often occurs with swapping "entering" vs. "exiting", or standing when you should be sitting.
Object/activity not visible. The activity or the object being used in the activity are not visible in the video. Please make sure that the video is well lit enough that the activity can be seen, and that the activity being performed is clearly visible in the video. If the review team cannot see it in the quicklook, then it will be flagged as not visible.
Repeated scene. Our review team has determined that you are using the same background again and again. This is a warning. We need new viewpoints and new scenes for your collections.
Awkward scene. Our review team has determined that the scene composition or audio contents has issues that make this a non-ideal submission. This is a warning. Review the collection requirements carefully.
Incorrect viewpoint. Our review team has determined that you are not collecting from the required camera viewpoint. The viewpoint is specified in the collection description. If the collection requires an elevated viewpoint, then the camera must be from the second story window or above.
Wrong location. The video was collected in the wrong location, such as indoors vs. outdoors. The location is specified in the collection description.
Invalid repetition count. This video was submitted without completing the required repetitions as defined in the description. A repetition is a repeat of the activity performed slightly differently each time, such that when each activity is performed the corresponding activity button is pressed to specify the activity start, the activity is performed with the button enabled, then when the activity is complete the button is disabled. The subject returns to a neutral body position with the activity button disabled, pauses, then repeats the activity. This review response typically means that either the collector performed only one repetition, or that the multiple repetitions were recorded but each repetition was not performed with separate activity button presses with pauses in between them.
Incorrect instances per video. The video did not contain the required number of activity instances in a single video. This often occurs when you are required to include a certain number of activities in a video, and this minimum number was not met.
Wrong Actor Pose. The video was collected with the actor not conforming to the required pose, such as sitting or standing. The required pose is specified in the description.
Incorrect Scene Illumination. The video was collected with the wrong lighting conditions, such as shadowed vs. direct sunlight. The required scene illumination is specified in the description.
Incorrect style. The video was collected with the actor not performing the the required style, such as an adverb like angry or excited. The required style is specified in the description.
Incorrect objects. The video was collected with the actor not interacting with the required object or objects. For example, the required object for Opening a microwave is a microwave. Required objects are specified in the description.
No joint activity. The video was collected without the actor performing a second joint activity performed simultaneously with the primary activity. An example is walking while talking on a cell phone, where the actor is performing the walking activity at the same time as talking on scell phone. The required joint activity is specified in the description.
Unusable video. The video submitted is not usable by the collector team. This is usually due to the video being too dark, your finger accidentally occluding the camera lens or contains a scene with nothing present (e.g. the sky or floor).
Corrupted video. The video file submitted to us is corrupted or unreadable. Please record and submit again.
Needs improvement. The video has something that requires secondary review, such as containing children under the age of 18. We require that all people contributing to the video are 18 years of age or older.
Consented subject not in video. Our review team has determined that the subject reported during consenting is not present in the video. When you are consenting your subject, you must enter the email address and record the consent video for the subject that will be in the video. The review team compares the consent video saved for this subject to the recorded person in the video, and if they are not the same, then the video will not be accepted. A common error is entering your email address during consent rather than the email address of your subject who will be recorded.
PII present. The video contains personally identifiable information (PII) of someone who has not consented to be in the collection. Examples include a recognizable face of a non-consented bystander, a readable license plate, or audio of a private conversation. Please re-record the collection in a location and at a time where no other people, vehicles or conversations can be captured incidentally, or arrange the framing so that any non-consented PII is out of the shot and out of the microphone's range.
Unrated. Our review team has not gotten to this submission yet, but it is in the queue for rating.
Under review. Our review team has not gotten to this submission yet, but it is in the queue for rating.
No GPS position recorded. Your submission did not include any GPS coordinates. Check that you have high accuracy or precise location GPS enabled for your device.
Incorrect GPS location. The GPS location reported with your submission does not match the required GPS location for this collection. Check that you collected at the right location.
Repeated GPS location. This location has already been submitted. Try to move to a new location at least 250 meters from this location.
Duplicate site. This site has already been submitted by another collector in your region. We have many collectors working with us, and some who share the same location. Try submitting a different site at least 1km away from here.
GPS accuracy too low. Your device is not receiving accurate GPS positioning from the available satellites. Avoid collecting near trees, bridges, tunnels or too close to buildings, or try again another time.
Low-quality audio. This collection requires audio, but the recording submitted to us has missing, unusable, or low-quality audio (e.g. silent, clipped, or with heavy background noise like a nearby motor, music, or other source). Make sure your microphone is not covered, you are not riding a motorcycle or car, you are not having a loud conversation or phone call while collecting, and that you are following the collection requirements for audible collections.
Video too short. The required duration in seconds as shown in the collection description is not met.
Outdated app. You are required to upgrade to the latest version available on the app store. This is required so that the additional metadata such as position, orientation and calibration data that we collect along with the video are accurately recorded. If your device does not support the minimum required version, then you will not be able to collect until you upgrade your device, or borrow a device from a friend (remember to log-in using your collector email!).
Collection max reached. You have submitted the maximum number allowable for this collection, and we cannot accept any more. This maximum is listed in the collection description.
Submission disabled. You are not currently recruited to this collection campaign. If you believe this is in error, contact the collector team.

How do I get paid?

We calculate payments based on the quality and quantity of the videos you submit to us, and the payments for the week are calculated on your collector dashboard.
Enter your payment provider (PayPal, Payoneer or Wise) and payment email in the app or log into your dashboard using the same credentials as the app, and update your payment details under Profile.
You can review your expected payments on your dashboard.
We support paying collectors via Payoneer, PayPal and Wise. Contact us at collector@visym.com if you would prefer alternative payment arrangements.

How do I use the dashboard at collector.visym.com?

Use the same email and password you use to log into the app. This will show your dashboard which summarizes your submissions and expected payments.
You may use "open in a new tab" button in the top right to access your dashboard as a standalone spreadsheet.
Enter your payment details under Profile. We currently support Payoneer, PayPal and Wise. You may also enter this payment information in-app.