DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1, 4-5, 12-16 and 18-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-2, 8, 11, 13 and 19-23 of U.S. Patent No. 12,198,277. Although the claims at issue are not identical, they are not patentably distinct from each other because claims 1, 4-5, 12-16 and 18-20 in the current application are broader than the reference claims 1-2, 8, 11, 13 and 19-23 of U.S. Patent No. 12,198,277.
Specifically, it is well established that “Omission of element and its function in combination is obvious expedient if remaining elements perform same functions as before” In re KARLSON (CCPA) 136 USPQ 184 (1963). Claims 1, 4-5, 12-16 and 18-20 in the current application are broader than the reference claims 1-2, 8, 11, 13 and 19-23 of U.S. Patent No. 12,198,277.
Below is a table indicating the corresponding relationship between claims 1, 4-5, 12-16 and 18-20 of the current application and claims 1-2, 8, 11, 13 and 19-23 of U.S. Patent No. 12,198,277.
Current Application
U.S. Patent No. 12,198,277
1
1
4
20
5
8
12
2
13
11
14
11
15
13
16
21
18
19
19
22
20
23
To perform analysis required, claim 1 of the current application is compared to claim 1 of U.S. Patent No. 12,198,277.
Claim 1: Current Application
Claim 1: U.S. Patent No. 12,198,277
A method comprising:
at an electronic device including one or more processors, a non-transitory memory, and a display:
obtaining a first semantic label value associated with a first object;
determining, based on the first semantic label value, a first display priority value associated with the first object;
prioritizing the first object over a second object based on the first display priority value;
and in response to determining that the first object satisfies an offscreen criterion, displaying, on the display, an offscreen indicator that is associated with the first object according to the prioritization.
A method comprising:
at an electronic device including one or more processors, a non-transitory memory, and a display:
while presenting, via the display, a plurality of objects including a first object and a second object:
obtaining a first semantic label value associated with the first object;
determining a first display priority value that is associated with the first object, wherein determining the first display priority value is based at least in part on the first semantic label value; and
prioritizing the first object over the second object based on a function of the first display priority value;
and in response to determining that each of the first object and the second object satisfies an offscreen criterion, displaying, on the display, a first offscreen indicator that is associated with the first object according to the prioritization.
As shown in the analysis above, claim 1 of the current application is broader than claim 1 of U.S. Patent No. 12,198,277. Therefore, this claim is properly subject to ODP rejection.
Similarly, ODP rejection can be shown for claims 4-5, 12-16 and 18-20 of the current application, as additional limitations in claims 4-5, 12-16 and 18-20 similarly recited in the reference claims 1-2, 8, 11, 13 and 19-23 of U.S. Patent No. 12,198,277.
Allowable Subject Matter
Claims 2-3, 6-11 and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
The present application is allowable based on the combination of elements defined by the claim language. Especially, the following limitation, in combination with other recited limitations, which the closest prior art of record taken either individually or in combination does not teach or suggest:
determining, based on the first semantic label value, a first display priority value associated with the first object.
The closest prior art of Bagnall et al. (US 2019/0188868) teaches a method comprising:
a display (display screen 205, fig. 5; [0024]: FIG. 5 is a display screen 205 showing targets 210, 220 in the field of view):
in response to determining that the first object satisfies an offscreen criterion (exiting the field of view of the display screen corresponds to offscreen criterion that both first target 210 and second target 220 satisfy; [0024]: As targets 210, 220 move, they may exit the field of view of display screen 205), displaying, on the display, an offscreen indicator (indicator 310) that is associated with the first object (fig. 6; [0025]: FIG. 6 shows the targets of FIG. 5 as they have veered off-screen in accordance with one embodiment of the present disclosure. The wider field of view shows the physical locations of targets 210, 220 after they have moved out of the frame of view of the display screen 205. In the embodiment shown in FIG. 6, the indicators 310, 320 are shown in the form of arrows in order to represent the direction of off-screen targets 210, 220. These indicators 310, 320 could further depict the distance from each of targets 210, 220 to the center of the display screen 205).
Another close prior art reference belonging to Godar et al. (US 2020/0174262) teaches an electronic device (computer, [0015]) including one or more processors (computer, [0015]; computer inherently comprises a processor), a non-transitory memory (non-transitory storage medium, [0016]), and a display ([0046]: HMD comprises a display portion 50); and obtaining a first display priority value (priority value of a virtual object) that is associated with the first object (A priority value can be associated with each virtual object in the image 2200; [0166]: Alternatively or in addition, the depth position of a virtual object and the priority value of that virtual object may be considered together so that a virtual object with a low priority value and a depth position near to the viewing point (such as the tree 2230) can be preferentially removed from the image 2200 before a virtual object with a low priority value and a depth position far away from the viewing point (such as the tree 2240). This means that a change in the information indicating the well-being of the user may mean that the tree 2230 is removed from the image 2200 while the tree 2240 is not removed, and a further change in the information may mean that the tree 2240 is subsequently removed from the image 2200. Similarly, a virtual object with a low priority value and a depth position far away from the viewing point (such as the tree 2240) can be preferentially removed from the image 2200 before a virtual object with a high priority value and a depth position far away from the viewing point (such as the avatar 2220). When simultaneously considering the depth position of a virtual object and the priority value of that virtual object, the box 2250 (with which the user may interact with) may have a higher priority value than the tree 2130, in which case the tree 2230 may be preferentially removed from the image 2200 before removing the box 2250 from the image; claim 13: the adapting step comprises adjusting either the number of virtual objects represented in the image or the texture resolution of each respective virtual object based on a priority value associated with each virtual object in the image).
Gregory et al. (US 11,753,029), another close prior art reference, teaches to prioritize the first object over a second object based the first display priority value (of the objects that are prioritized as detectable by the system, only detectable objects that the vehicle is responding to are represented with off-screen notifications; col. 10 line 50 – col. 11 line 5: In some examples, the method may include refraining from causing presentation of a visual indicator associated with an object that is disposed outside of the first field-of-view. For instance, based at least in part on the sensor data, an attribute associated with a detected object may be determined and, based at least in part on the attribute, the user interface may refrain from presenting a visual indicator that is associated with the object. In examples, the attribute may include one or more of a trajectory of the object, a speed of the object, a proximity of the object to the first field-of-view, an object type associated with the object, or a predicted time until the object intersects with the first field-of-view. By way of example and not limitation, the user interface may refrain from presenting the visual indicator if a trajectory of the object does not intersect with the first field-of-view. Additionally, or alternatively, based at least in part on the sensor data, a classification of a detected object may be determined and, based at least in part on the classification, the user interface may refrain from presenting a visual indicator that is associated with the object. In some examples, a classification of an object may include whether a car is parked or mobile, whether a pedestrian is stationary, walking, running, entering a crosswalk, etc., and the like; col. 27 line 66 – col. 28 line 1: In some examples, only detectable objects that the vehicle may be responding to may be represented with off-screen notifications 808; col. 35 lines 40-47: detecting, based at least in part on the data, a second object in the environment; determining, based at least in part on the data, that the second object is disposed outside of the first field-of-view; determining, based at least in part on the data, an attribute associated with the second object; and based at least in part on the attribute, refraining from causing presentation, on the user interface, of a visual indicator that is associated with the second object); and displaying, on the display, a first offscreen indicator that is associated with the first object (off-screen notification 808 is displayed for detected object 806) according to the prioritization (col. 27 line 66 – col. 28 line 22: In some examples, only detectable objects that the vehicle may be responding to may be represented with off-screen notifications 808. For instance, the vehicle 102 may detect that another vehicle 810 is located within the second field-of-view 804. However, an off-screen notification 808 may not be presented on the user interface 800 to indicate the presence of the other vehicle 810 because the direction of travel (e.g., trajectory) of the other vehicle 810 is moving away from the vehicle 102. In some examples, the off-screen notifications 908 may indicate the direction in which detected objects 806 and/or 814 are approaching the vehicle 102 from, a type of object associated with the detected objects 806 and/or 814 (e.g., whether the detected objects are a vehicle, pedestrian, bicyclist, etc.), an estimated amount of time until the detected objects 806 and/or 814 are within the first field-of-view 802 and/or the canvas 816, a position on the user interface 800 in which the detected objects 806 and/or 814 are predicted to first appear, and the like. In some examples, whether off-screen notifications 808 are presented on the user interface 800 may be based at least in part on a speed determined for the detected objects 806 and/or 814, an amount of time until the detected objects 806 and/or 814 enter the first field-of-view 802, and/or a size of the detected objects 806 and/or 814).
Salter et al. (US 2017/0069143) teaches the first object represents a physical object ([0045]: FIG. 8 also depicts a real object in the form of a keyboard 808), and wherein determining that the first object satisfies the offscreen criterion includes performing a computer-vision technique (using logic subsystem and storage subsystem of the controller for identifying real objects in the augmented reality field of view) with respect to the first object ([0023]: Display system 300 further comprises a controller 320 having a logic subsystem 322 and a storage subsystem 324 in communication with the sensors, gaze detection subsystem 310, display subsystem 304, and/or other components. Storage subsystem 324 comprises instructions stored thereon that are executable by logic subsystem 322, for example, to receive and interpret inputs from the sensors, to identify location and movements of a user, to identify real objects in an augmented reality field of view and present augmented reality imagery therefore, to detect objects located outside a field of view of the user, and to present indications of positional information associated with objects located outside the field of view of the user, among other tasks; [0039]: For example, if a user is currently entering text into a text entry box and then gazes in a different direction such that the text box is no longer visible, the display of a tendril that leads back to the text box may be triggered; the system determines a keyboard as a real object displayed in the augmented reality field of view of the user, and when the user changes his focus from the keyboard to a different direction such that the keyboard moves out of the field of view of the user and therefore is not being displayed, a visual indicator such as a tendril will be displayed that leads the user back to the keyboard); determining that at least one of the first object (text box) and the second object satisfies the offscreen criterion (text box moves out of the field of view of the user when the user gazes in a different direction) based on the positional change (different gaze direction) input ([0022]: Display system 300 may further include one or more motion sensors 318 to detect movements of a user's head when the user is wearing display system 300. Motion data may be used, potentially along with eye-tracking glint data and outward-facing image data, for gaze detection, as well as for image stabilization to help correct for blur in images from the outward-facing image sensor(s) 306. The use of motion data may allow changes in gaze location to be tracked even if image data from outward-facing image sensor(s) 306 cannot be resolved. Likewise, motion sensors 318, as well as microphone(s) 308 and gaze detection subsystem 310, also may be employed as user input devices, such that a user may interact with the display system 300 via gestures of the eye, neck and/or head, as well as via verbal commands; [0039]: For example, if a user is currently entering text into a text entry box and then gazes in a different direction such that the text box is no longer visible, the display of a tendril that leads back to the text box may be triggered. A tendril also may lead a user to objects that are obscured by other objects; [0038]: A tendril may be used in any suitable context to lead a user's gaze to an object located outside of a current augmented reality field of view. The tendril acts as a line that leads from within the field of view toward the object outside the field of view. Thus, a user may visually follow the tendril to find the object to which the tendril leads; [0040]: A tendril may have any suitable appearance. For example, in some embodiments a tendril may have a vine-like appearance. In other embodiments, the tendril may have the appearance of any other suitable representation of a path leading to an out-of-view object. A tendril may originate from a location adjacent to a periphery of a display, or may originate closer to a center of a user's field of view); and moving the first offscreen indicator to a different position on the display based on the positional change input (fig. 6 shows a change in the position of the visual indicator 604 when the user changes his/her gaze direction; [0037]: FIG. 6 shows another example embodiment of a visual indicator. In this example, a room environment 602 is shown in field of view 102. At time T1, a visual indicator 604 is displayed at a bottom right periphery of field of view 102 to indicate the presence of an object located outside field of view 102 in a direction outside the bottom right of field of view 102. The visual indicator 604 may comprise a route, a path, an arrow, or other indication of directionality which a user may follow in order to be directed to the object it represents. In the depicted embodiment, visual indicator 604 may be represented as a tendril (e.g. a vine-like feature or other representation of a line) extending from object 606, which may represent a notification block or other virtual object. Thus, a user may visually follow indicator 604 towards object 606 by turning towards the right to bring the object 606 into view; [0038]: A tendril may be used in any suitable context to lead a user's gaze to an object located outside of a current augmented reality field of view. The tendril acts as a line that leads from within the field of view toward the object outside the field of view. Thus, a user may visually follow the tendril to find the object to which the tendril leads).
Chalmers et al. (US 2021/0365113) teaches determining that each of the first object and the second object satisfies the offscreen (object outside of the field of view of the display; the field-of-view of a display is a range of a scene that is visible on a display screen, and therefore anything that is out of the field of view of the display corresponds to being outside the edges of the display screen) criterion includes: determining that at least a portion of the first object (object 806) is positioned at less than a threshold distance from a first edge (field of view) of the display (object outside of the field of view of the display); and determining that at least a portion of the second object is positioned at less than a threshold distance from a second edge of the display ( [0149]: FIG. 11B illustrates device 202 at a time that corresponds to the scenario depicted in FIG. 11A. With reference to FIGS. 11A and 11B, assume device 202 updates not only the LEDs but also secondary display 206 because device 202 is closer (e.g., within a set of predetermined threshold distances) at distance 804d away from object 806 (as opposed to device 202 being distance 804c away from object 806 in FIG. 10B); [0152]: In some embodiments, because device 202 is distance 804d away from object 806 and/or object 806 is within a predetermined threshold distance outside the field-of-view of primary display 204, device 202 causes secondary display 206 to display modified representation 826a of object 806. Here, modified representation 826a is a representation of a proxy object or a symbolic representation of object 806 and is visually distinguishable from object 806. As illustrated in FIG. 11B, modified representation 826a is a blob of a plurality of pixels at the edge of secondary display 206; [0154]: In some embodiments, object 806 is outside of the field-of-view of secondary display 206 when device 202 is distance 804d away from object 806. Thus, in some embodiments, device 202 displays modified representation 826a, via secondary display 206, when object 806 is outside of the field-of-view of secondary display 206).
El Essaili et al. (US 11,202,117) teaches obtaining one or more user preference criteria (claim 1: obtaining user preference information which indicates which of the spatial objects is of higher relevance to the user than other spatial objects); and determining the first display priority value based on a function of the first object and the one or more user preference criteria (claim 1: for at least the first spatial object, assigning a priority value to the first spatial object based on the obtained user preference information).
Lee (US 2012/0154619) teaches obtaining the first display priority value (priority is based on combined the probability of an object being there and the probability map of the eyes) includes determining a first engagement score (probability map based on the eyes) that characterizes a level of user engagement (based on eyes of the user) with respect to the first object ([0046]: In order to aid in the object identification process, AR unit 25 may utilize the information provided by the position or pupil direction of the eyes of the user (which are captured by second video camera 22). First video camera 21 may capture a first image (or a first sequence of images), and first video camera 21 may be pointed outward relative to a user of video device 20. Second video camera 22 may capture a second image (or a second sequence of images), and the second video camera may be oriented to capture the eyes of the user when the first video camera 21 is pointed outward relative to the user. AR unit 25 may generate a probability map based on the eyes of the user in the second image (or the second sequence of images), and may generate AR information associated with one or more objects in the first image (or the first sequence of images) based at least in part on the probability map; [0059]: Consistent with the techniques of this disclosure, generation of AR information 63 may be more likely than generation of AR information 64. This should be apparent from the combined probability map of FIG. 5, where areas 51 and 55 have higher priority values than areas 56, 57, 58, 59, 61 and 62. The higher priority areas may be processed before lower priority areas, and therefore, given a limited amount of power, time or processing capabilities, the corresponding areas of an image that correspond to the higher priority areas of the probability map may be more likely to be overlaid with AR information. In this example, the area around object 1 has the highest priority due to the combined probability of an object being there (as defined in the first probability map of FIG. 4A) and the high probability assigned based on the eyes of the user (as defined in the second probability map of FIG. 4B).
Chiu et al. (US 11,698,677) teaches determining the first engagement score is based on extremity tracking data that is indicative of an extremity position (col. 7 lines 38-46: The extremity tracking sensor 150 obtains extremity tracking data indicative of a position of an extremity of a user. For example, in some implementations, the extremity tracking sensor 150 corresponds to a hand tracking sensor that obtains hand tracking data indicative of a position of a hand or a finger of a user within a particular object. In some implementations, the extremity tracking sensor 150 utilizes computer vision techniques to estimate the pose of the extremity based on camera images), including determining a spatial relationship between the extremity position and the first object (col. 7 lines 38-46: The extremity tracking sensor 150 obtains extremity tracking data indicative of a position of an extremity of a user. For example, in some implementations, the extremity tracking sensor 150 corresponds to a hand tracking sensor that obtains hand tracking data indicative of a position of a hand or a finger of a user within a particular object. In some implementations, the extremity tracking sensor 150 utilizes computer vision techniques to estimate the pose of the extremity based on camera images; col. 9 lines 38-64: As illustrated in FIG. 2D, in some implementations, the electronic device 210 includes an extremity tracker 240 (e.g., the extremity tracking sensor 150 in FIG. 1) that provides extremity tracking data associated with the user 50. The extremity tracking data indicates a position of a user's extremity (e.g., finger(s), hand, foot). For example, as illustrated in FIG. 2D, based in part on the extremity tracking data, the electronic device 210 determines that the free hand 52 of the user 50 is at a location corresponding to the bottom of the book 220. The free hand 52 is beginning a left swipe gesture in order to request the electronic device 210 to replace the first text content 222-1 with second text content 222-2 (e.g., turn the page to the second page of the book 220). The left swipe gesture is indicated by movement line 242 (illustrated for purely explanatory purposes). As illustrated in FIG. 2E, the free hand 52 is approximately halfway finished with the left swipe gesture, and the free hand 52 completes the left swipe gestures in FIG. 2F. In response to detecting termination of the left swipe gesture, the electronic device 210 replaces the first text content 222-1 with the second text content 222-2, as illustrated in FIG. 2F. The second text content 222-2 corresponds to the second page of the “Blue the Dog” story. Based on the left swipe gesture, the electronic device 210 increases the engagement score 224 from “5” to “7” because the electronic device 210 determines (e.g., infers) that the user 50 intends to continue to read the book 220; col. 13 lines 27-33: For example, with reference to FIGS. 2J-2L, the first dog 250 moves from the initial location of the display 212 to a destination location on the display 212 that is closer to the book 220. As another example, the movement corresponds to extremity movement of the second object, such as an individual waving hands in order to get the attention of the user of the electronic device).
Garipov (US 2017/0052766) describes semantic metadata is stored as data values, which are obtained at runtime and displayed in labels. The semantic metadata can take any form, for example natural language, i.e. “Balance due,” or it can be any data value, including values derived at run time, such as a time-and-date stamp for the run of an instrument or a software file, an image or icon, or a reference to a portion of a sequence which is being computed successively.
Abhyankar et al. (US 2020/0250245) describes a semantic graph service 120 can combine the general weights 132 with the opinion measures to produce opinion-customized weights 134 to generate a response to the query, which is provided to the server system 110. The response may identify objects stored in the database 112 and may include scores for or a ranking of objects, as well a potentially other information (e.g., semantic tags or labels, associated keywords, object type definitions, object attributes, etc.).
Xia et al. (US 2021/0201038) describes a video content provider or video providing user may increase the label weights of the video labels in one or more semantic recognition dimensions or decrease the label weights of the video labels in one or more semantic recognition dimensions based on the collected video preferences of the user. In some embodiments, the increase or decrease of a label weight of a video label is reflected in candidate label combinations displayed to the user. If the label weight of the video label is increased, a display priority of the candidate label combination including the video label corresponding to the label weight is accordingly raised; on the contrary, if the label weight of the video label is decreased, the display priority of the candidate label combination including the video label corresponding to the label weight is accordingly lowered.
Lecue et al. (US 2021/0264226) describes the object detection task includes comparing the object detection confidence score to a threshold and comparing the object detections to the extracted semantic links between labels when the confidence score is greater than the threshold.
Kumar et al. (US 11301684) describes process for assigning final semantic labels to pixel values of a video frame by analyzing semantic values applied to pixel groups from frames prior to and subsequent to the video frame. Upon receiving the video data, the event-determination component may input each frame of the video data into a semantic classifier that is configured to provide semantic labels to each pixel of each frame or to detect regions of interest within the frame. As used herein, a semantic label is used to indicate what the pixel data of the frame represents. For instance, example values of a semantic label may include “background”, “shelf”, “person”, “item”, and/or the like. Pixel data having a semantic-label value of “background” may indicate that the corresponding pixel data illustrates background, such as a wall, floor, or the like. Pixel data having a semantic-label value of “person” may indicate that the corresponding pixel data illustrates a user. After training, the semantic classifier may be configured to receive video data and determine a grid of probabilities indicating a probability that each pixel of a frame or over multiple frames depicts a respective semantic-label value (e.g., head, background, etc.). Given that subsequent operations of the techniques may utilize one defined semantic-label value per pixel (over one or more multiple frames of the video data), this probability grid may be converted into a discrete segmentation mask.
Zeiler et al. (US 10750245) describes a user interface 200 may include one or more user interface elements and/or portions. The user interface elements and/or portions may comprise one or more of: a video canvas 201 for rendering video content for playback; a navigation portion 202 for navigating through time within the current video; a label filter portion 203 conveying of a subset of labels relevant to the current video, scene, and/or view zoom levels shown in the video canvas 201; a time series portion 204 depicting the time series of label association confidence for a subset of labels (e.g., shown in the label filter portion 203) over the duration of the video, scene, and/or view zoom level; a label selection portion 300 (FIG. 3) for changing a current selection of labels 210 (and displayed in the labels filter portion 203); a search portion 206 for receiving user entry and/or selection of information related to a query for searching for semantic labels within the video; a similar scene display portion 207 showing a set of videos or scenes ranked by similarity to the current video, scene, or view zoom level (e.g., based on similar content, semantic labels, and/or other considerations); and/or other elements and/or portions.
Richter (US 2022/0012283) describes a method includes obtaining a first unstructured video stream that provides pixel values for a plurality of pixels and corresponds to a portion of a second unstructured video stream being displayed on a second electronic device different from the first electronic device. Obtaining the first unstructured video stream includes obtaining pass-through image data including the portion of a second unstructured video stream. The method includes generating respective pixel characterization vectors for a first portion of the plurality of pixels. Generating each of the respective pixel characterization vectors includes determining a respective instance label value. The method includes identifying a first object within the first portion of the plurality of pixels associated with a particular instance label value. The method includes generating respective semantic label values corresponding to pixels associated with the first object. The respective semantic label values are added to pixel characterization vectors associated with the first object. The first electronic device 201 generates respective semantic label values corresponding to pixels associated with the first object. The respective semantic label values are added to pixel characterization vectors associated with the first object. In some implementations, the first electronic device 201 appends respective semantic label values to the pixel characterization vectors associated with the first object. Continuing with the previous example, the first electronic device 201 generates respective semantic label values of “Golden Retriever Dog” for the dog 240, as indicated by the corresponding semantic label value identifier 240b in FIG. 2D. One of ordinary skill in the art will appreciate that in some implementations, the first electronic device 201 generates and displays a different or additional corresponding semantic label value identifier 240b (e.g., “Furry Friend”) and/or foregoes displaying the corresponding semantic label value identifier 240b altogether.
Yoon et al. (US 2001/0056427) describes a method for browsing a multimedia according to the present invention, includes selecting a semantic element or segment information, selecting a link information of the selected semantic element or segment, and displaying corresponding segment or semantic element information in sequence according to the priority/weight between the corresponding segment element and segment which was obtained by the link information, wherein a multimedia data includes a semantic element structure describing a content of a multimedia and a segment information structure of the multimedia for browsing the multimedia based on content. Using the priority/weight information of the link information 403, the semantic elements can be displayed in order from the highest to the lowest priority/weight, and scenes or shots of the segment describing the corresponding semantic elements may be displayed, in sequence, from the highest to the lowest priority.
Luo et al. (US 2021/0398353) describes that during the training of the deep learning network, pixel-level labeling (e.g., semantic labeling or plane labeling) can be performed on the training sample. The semantic labeling uses a semantic object as a unit to label this semantic object at a specific position in the multimedia information (the multimedia information will be described by taking an image as example hereinafter); and the pixel-level semantic labeling is to ensure the specific position accurate to a pixel level and semantically label each pixel point in an image serving as the training sample. For example, if a vehicle is selected as a semantic object, all pixel points belong to the vehicle in an image are labeled as identical semantic information (e.g., a sematic attribute identifier).
Tsukagoshi (US 2019/0174151) describes when one or a plurality of regions is selected by user operation, image data for display for each display unit is obtained so that each of objects included in the selected regions is displayed in a display unit at a position corresponding to its degree of priority on the basis of display priority information of the region obtained by the semantic region SEI analysis unit 264.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JWALANT B AMIN whose telephone number is (571)272-2455. The examiner can normally be reached Monday-Friday 10am - 630pm CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at 571-272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JWALANT AMIN/Primary Examiner, Art Unit 2612
Prosecution Insights