DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: “body model generation system 230” in paragraphs 26-27 and “corresponding virtual object 1022” in paragraph 138. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claim 15 objected to because of the following informalities: typo wherein, “the 3d position” in claim 15 should read “a 3d position”. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 3, 14, 18 and 20 rejected under 35 U.S.C. 101 because the claimed invention is
directed to an abstract idea without significantly more.
MPEP 2106 Ill provides a flowchart for the subject matter eligibility test for
product and processes. The claim analysis following the flowchart is as follows:
Regarding claim 1:
Step 1: Is the claim to a process, machine, manufacture or composition of matter?
Yes. It recites a method, which is a process.
Step 2A, Prong One: Does the claim recite an abstract idea, law of nature, or
nature phenomenon?
Yes.
Regarding independent claim 1, the claim recites “A method comprising: identifying a given point of multiple points of an object that moves less relative to other points of the multiple points; selecting, as a reference point, the given point that has been identified as moving less relative to the other points; and tracking movement of the object using the selected reference point.”
The limitation “identifying a given point…” as drafted, is a process that, under its
broadest reasonable interpretation, is directed to a mental process that can be processed either by
a pen and paper or a person is capable of act on it in the mind. For example, the limitation in the
context of this claim encompasses a situation where a person can mentally identify a point that moves less compared to other points of an object.
Similarly, the limitation of “selecting, as a reference point…”, as drafted, is a process that,
under its broadest reasonable interpretation, is directed to a mental process that can be processed
either by a pen and paper or a person is capable of act on it in the mind. For example, the
limitation in the context of this claim encompasses a situation where a person can mentally take
the aforementioned identified given point and mentally select/choose it as a reference point.
Similarly, the limitation of “tracking movement of the object using the selected reference point”, as drafted, is a process that, under its broadest reasonable interpretation, is directed to a mental process that can be processed either by a pen and paper or a person is capable of act on it in the mind. For example, the limitation in the context of this claim encompasses a situation where a person can mentally take the aforementioned selected/chosen reference point and track object from it.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the
limitation in the mind but for the recitation of generic method, then it falls within the “Mental
Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A, Prong Two: Does the claim recite additional elements that integrate the judicial exception into a practical application?
No.
The claim does not recite any additional elements except the limitations
identified as abstract idea (mental process) in Step 2A Prong One.
Therefore, this judicial exception is not integrated into a practical application
because no additional elements other than the abstract idea limitations.
Step 2B: Does the claim recite additional elements that amount to significantly
more than the judicial exception?
No.
The claim does not include additional elements that are sufficient to amount to
significantly more than the judicial exception because as discussed above with respect to
integration of the abstract idea into practical application. The claim is not patent eligible.
Regarding dependent claim 3, the claim recites “further comprising: determining a size of a body part corresponding to the given point of the multiple points of the object depicted in a video; identifying a portion of a skeletal rig of a virtual item that corresponds to the body part; and modifying a size of the portion of the skeletal rig to match the determined size of the body part corresponding to the given point of the multiple points of the object depicted in the video.” Determining and modifying size involves mathematical calculations and identifying here would be a mental process similar to the above claim 1. Therefore, claim 3 recites abstract idea without additional elements. Similar to the discussion above with respect to claim 1, no additional elements are recited to integrate the abstract idea into practical application or amount to significantly more than the abstract idea.
Regarding dependent claim 14, the claim recites “further comprising: determining a size of the object depicted in a video; and adjusting a size of a virtual item based on the size of the object.” Determining and adjusting size involves mathematical calculations. Therefore, claim 14 recites abstract idea without additional elements. Similar to the discussion above with respect to claim 1, no additional elements are recited to integrate the abstract idea into practical application or amount to significantly more than the abstract idea.
Claims 18 and 19 recite similar limitations discussed above with respect to
claim 1 but with additional elements of system with processors to perform operations. The system with processor(s) are generic computer component that do not integrate the abstract ideas recites in these claims into practical application or amount to significantly more (see MPEP 2106.05(a), (b), and (f)).
Claim 20 recite similar limitations discussed above with respect to
claim 1 but with additional elements of non-transitory computer-readable
medium that can cause the processor to perform operations. The non-transitory
computer-readable media and processor are generic computer component that do not
integrate the abstract ideas recites in these claims into practical application or amount
to significantly more (see MPEP 2106.05(a), (b), and (f)).
Therefore, claims 1, 3, 14, 18 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract idea without significantly more.
Regarding claims 2 and 19, recite that “causing to be displayed… updating the 3D position of the virtual item in the video…”. This makes it to where the displaying and updating is interpreted as rendering of an image. Therefore, these claims overcome the 101 rejections of their parent claims, due to additional elements that can integrate the abstract ideas into practical application (rendering).
Regarding claims 4, 9-13, and 15-16 recite that “moving a virtual item… turning a virtual item… updating a 3D position of a virtual item…displaying a first virtual item…second virtual item is placed… updating a position of a second virtual item… generate a plurality of identical virtual items”. This makes it to where the moving/turning/updating/displaying/placing and generating (of virtual item) is interpreted as rendering of an image. Therefore, these claims overcome the 101 rejections of their parent claims, due to additional elements that can integrate the abstract ideas into practical application (rendering).
Regarding claims 5-8 and 17 they depend on and incorporate matter from claim 4, this makes it to where the updating/comparing position and switching frames is interpreted as rendering of an image. Therefore, these claims overcome the 101 rejections of their parent claims, due to additional elements that can integrate the abstract ideas into practical application (rendering).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4, 12, 14-15, and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Monastyrshyn (U.S. Patent Application Publication No. 2019/0196663), hereinafter referenced as Monastyrshyn in view of Nebehay (Clustering of Static-Adaptive Correspondences for Deformable Object Tracking), hereinafter referenced as Nebehay.
Regarding claim 1, Monastyrshyn teaches A method comprising: and tracking movement of the object using the selected reference point (paragraph 43 teaches “determine the object of interest based on a size, a position, a prominence, a shape, a color, changes in color, a set of reference points, edge detection operations, object tracking operations, or any other suitable definable or detectable aspects of the object”); the object tracking operations here would be done using the set of reference points, set can contain a single value such as single reference point, and for a point to become reference point (and be used) it must be selected.
However, Monastyrshyn fails to teach identifying a given point of multiple points of an object that moves less relative to other points of the multiple points; selecting, as a reference point, the given point that has been identified as moving less relative to the other points;
However, Nebehay teaches identifying a given point of multiple points of an object that moves less relative to other points of the multiple points (Nebehay, page 1, right hand column (RHC) second paragraph teaches “second contribution is a novel method for establishing correspondences based on the insight that static and adaptive correspondences complement each other as they stem from opposite ends of the adaptivity spectrum” and page 3, left hand column (LHC) fourth paragraph teaches “we overrule adaptive correspondences by
static ones when both models yield a result as the latter are not affected by drift”); static point/correspondence identified here is the given point of multiple/adaptive points of an object, and the static point moves less relative to the adaptive/multiple points since it is static; selecting, as a reference point, the given point that has been identified as moving less relative to the other points (Nebehay, fig. 1 description teaches “From the initial bounding box in frame t0 static keypoints are extracted. Both the static keypoints and adaptive keypoints from frame t − 1 are matched to the current frame t.”); this shows static keypoint taken as a reference point by being extracted and it is moving less than the adaptive points since it is static. Nebehay is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of identifying point that moves less relative to other points in the process of object tracking. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Monastyrshyn's invention with the selecting static point techniques of Nebehay to exploit the parallelizability
of the clustering and the keypoint matching to improve computation time (Nebehay, page 6, RHC, last paragraph). This is done due to using stable/static points and rejecting outlier/bad points which also means using less points thus faster computation.
Regarding claim 2, the combination of Monastyrshyn and Nebehay teaches further comprising: computing a 3D position for placement of a virtual item relative to the reference point (Monastyrshyn, paragraph 47 teaches “facial reference points representing one or more feature”, paragraph 48 teaches “movement component 240 may determine a first position for the reference feature 408 in a first frame (e.g., FIG. 4). The reference feature 408 may be one or more of a set of landmarks, a set of edges, or any other distinguishing aspect of the object of interest 402” and paragraph 49 teaches “a body or avatar including the object of interest about the graphical user interface”); reference point represents feature here thus the determining/computing of 3d position/placement here of reference feature (which corresponds to augmented reality/virtual item since is included in avatar) is relative to such; causing to be displayed the virtual item within a video at the 3D position (Monastyrshyn, paragraph 50 teaches “As shown in FIGS. 4-6, the augmented reality component 230 and the movement component 240 track the first augmented reality element 406 and the object of interest 402 within the frames or images comprising the video stream”); this shows the virtual/augmented reality item would be displayed within video at the aforementioned 3D position; and updating the 3D position of the virtual item in the video as the reference point changes based on the movement of the object (Monastyrshyn, paragraph 49 teaches “object of interest may interact with a portion of the augmented reality elements (e.g., the first augmented reality element) by changing positions of the object of interest to avoid, contact, catch, jump over, or otherwise perform an action or set of actions relative to at least one augmented reality element. For example, the object of interest may be presented within an augmented reality environment and depicted as a person skiing down a hill. Movement of the object of interest (e.g., the face) may control movement of a body or avatar including the object of interest about the graphical user interface to avoid obstacles (e.g., a portion of augmented reality elements), collect or contact goals (e.g., a portion of augmented reality elements), and proceed down a rendered ski slope”); this shows updating 3D position of virtual item/avatar as object of interest (and reference point thereof) changes/moves.
Regarding claim 4, the combination of Monastyrshyn and Nebehay teaches further comprising: determining that the object has moved from a first 3D position to a second 3D position between a first frame and a second frame of a video (Monastyrshyn, paragraph 48 teaches “movement component 240 may then determine or detect a second position for the object of interest 402 or the reference feature 408 in a second frame (e.g., FIG. 5)); this shows determination of object moving from first to second position between first and second frame; computing a distance and trajectory of the movement of the object from the first 3D position to the second 3D position (Monastyrshyn, paragraph 68 teaches “augmentation component 230 may modify presentation of the first augmented reality element 808, relative to the object of interest 802 or a portion thereof, in one or more frames of the video stream. In FIGS. 8-10, the first augmented reality element 806 is depicted as traveling along a trajectory across a set of frames”); modifying representation and traveling of first augmented reality element (virtual item) alongside trajectory of such (with both being relative to object of interest) shows that object’s distance and trajectory of movement must be computed which is done for the aforementioned first 3D position to second 3D position movement; and moving a virtual item from a third 3D position to a fourth 3D position based on the distance and trajectory of the movement of the object from the first 3D position to the second 3D position (Monastyrshyn, paragraph 55 teaches “the first augmented reality element 808 may be presented at a first position and depicted as following a trajectory, path, or otherwise change position from the first position to a second position”); the virtual item here moving from first to second position is considered moving a virtual item from a third 3D position to a fourth 3D position (because the first two positions are for object) and this is based on distance and trajectory of movement of object from the first 3D position to the second 3D position because as aforementioned the first augmented reality element (virtual item) has its presentation modified relative to the object of interest.
Regarding claim 12, the combination of Monastyrshyn and Nebehay teaches comprising: displaying a first virtual item (Monastyrshyn, paragraph 38 teaches “augmented reality elements within a graphical user interface presented on a display component of the client device); this show augmented reality element (virtual item inclusive of first) displayed; and causing to be displayed a second virtual item that mimics movement and placement of the first virtual item (Monastyrshyn, paragraph 70 teaches “the second augmented reality element 1010 may be presented as falling along a similar trajectory or direction as the first augmented reality element”); second augmented reality element (second virtual item) falling along similar trajectory as first shows it would mimic movement and placement of the first virtual item when being displayed.
Regarding claim 14, the combination of Monastyrshyn and Nebehay teaches further comprising: determining a size of the object depicted in a video (Monastyrshyn, paragraph 43 teaches “recognition component 220 may determine the object of interest based on a size”); this means that size of object depicted in video must be determined since object of interest is determined based on size; and adjusting a size of a virtual item based on the size of the object (Monastyrshyn, paragraph 65 teaches “change a size of the first augmented reality element); changing/adjusting size of virtual item here is based on size of object since comes in a step after object of interest is determined based on size (since previous steps here rely on object of interest).
Regarding claim 15, the combination of Monastyrshyn and Nebehay teaches wherein the reference point is a first 3D reference point and the given point is a first point, (Monastyrshyn, paragraph 43 teaches “may determine the object of interest based on a size, a position, a prominence, a shape, a color, changes in color, a set of reference points,”, paragraph 81 teaches “depth position of the object of interest” and Nebehay, page 1, fig. 1 description teaches “From the initial bounding box in frame t0 static keypoints are extracted.”); static keypoint (given point) here is first point since comes from initial bounding box in frame t0, set of reference points would include a first reference point, and depth position indicates the reference point would be at first 3D position; and wherein a virtual item is a first virtual item, (Monastyrshyn, paragraph 45 teaches “augmentation component 230 may then present a first augmented reality element 406 within the graphical user interface 400 along with the object of interest 402”); this shows virtual item as a first virtual item; further comprising: selecting a second 3D reference point corresponding to a second point of the multiple points of the object depicted in a video (Monastyrshyn, paragraph 43 teaches “may determine the object of interest based on a size, a position, a prominence, a shape, a color, changes in color, a set of reference points,” and Nebehay, fig. 1 description teaches “From the initial bounding box in frame t0 static keypoints are extracted. Both the static keypoints and adaptive keypoints from frame t − 1 are matched to the current frame t”); in a scenario, set of reference points from Monastyrshyn would contain plural (set thus has second) reference points, the static keypoints (plural) extracted from Nebehay indicates a second point of multiple points that the reference point would correspond to, for a point to become reference point (and be used) it must be selected and this would be 3D point due to same reasoning as above of depth mentioned in Monastyrshyn for the object; and updating a position of a second virtual item based on changes to the second 3D reference point independently of updating the 3D position of the first virtual item based on changes to the first 3D reference point (Monastyrshyn, paragraph 70 teaches “second augmented reality element 1010 may initially occupy a distinct position as the first augmented reality element”, paragraph 87 teaches “second augmented reality element… may be distinct from the first augmented reality element”); this shows in some embodiments that second virtual item position wouldn’t depend on first virtual item position, thus updating position of second virtual item based on changes to the aforementioned second 3D reference point would be done independently of updating the 3D position of the first virtual item based on changes to the aforementioned first 3D reference point.
Regarding claim 18, the system claim 18 recites similar limitations as method claim 1, and thus is rejected under similar rationale. In addition, Monastyrshyn, fig. 15 teaches system/device 1500 with processor 1510 to perform operations.
Regarding claim 19, the system claim 19 recites similar limitations as method claim 2, and thus is rejected under similar rationale.
Regarding claim 20, the method claim 20 recites similar limitations as method claim 1, and thus is rejected under similar rationale. In addition, Monastyrshyn, claim 11 teaches “a non-transitory processor-readable storage medium coupled to the one or more processors, the non-transitory processor-readable storage medium storing processor executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising”.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Monastyrshyn and Nebehay as applied to claim 1 above, and further in view of Paulovich (U.S. Patent Application Publication No. 2016/0210780), hereinafter referenced as Paulovich.
Regarding claim 3, the combination of Monastyrshyn and Nebehay fails to teach
further comprising: determining a size of a body part corresponding to the given point of the multiple points of the object depicted in a video; identifying a portion of a skeletal rig of a virtual item that corresponds to the body part; and modifying a size of the portion of the skeletal rig to match the determined size of the body part corresponding to the given point of the multiple points of the object depicted in the video.
However, Paulovich teaches further comprising: determining a size of a body part corresponding to the given point of the multiple points of the object depicted in a video (Paulovich, paragraph 3 teaches “workpiece appearing in a one-to-one size ratio with a size of the user in the real world” and paragraph 68 teaches “step 612, the system may detect and track a user's skeleton and/or hands as described above, and update the real world model based on the positions of moving body parts and other moving objects); this shows determining size of body part such as skeleton and/or hands of object/user (because having 1:1 ratio with size of user in real world requires it) and when viewed in combination, this would be the object from claim 1 thus corresponds to given point of multiple points in a video (the video mentioned in Monastyrshyn, paragraph 50); identifying a portion of a skeletal rig of a virtual item that corresponds to the body part (Paulovich, paragraph 59 teaches “to track the position of users within a scene, users may be recognized from image data. The processing unit 4 may implement a skeletal recognition and tracking module 448” and paragraph 60 teaches “may further include a gesture recognition engine 454 for receiving skeletal model”); this shows skeletal model received and recognition module which shows identifying/recognizing a portion of skeletal rig of virtual item/model that corresponds to the user recognized from image data (thus corresponds to the body part); and modifying a size of the portion of the skeletal rig to match the determined size of the body part corresponding to the given point of the multiple points of the object depicted in the video (Paulovich, paragraph 3 teaches “workpiece appearing in a one-to-one size ratio with a size of the user in the real world” and paragraph 65 teaches “the height of a user may be used to determine a scaling ratio of the avatar once sized and placed in a virtual content”); determining scaling ratio of avatar (previously mentioned as skeletal model) shows modifying size of portion of skeletal rig and since for a 1:1 ratio, this is done to match size of body part (which is corresponding to given point of multiple points of object depicted in the video as aforementioned, cited above and when viewed in combination). Paulovich is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of scaling virtual item to match object in video size. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Monastyrshyn and Nebehay's invention with the identifying skeletal rig and body parts alongside sizing techniques of Paulovich to ensure the experience is optimized (paragraph 89). This would be due to the modifying size as aforementioned for a more realistic and engaging experience.
Claim(s) 5-9 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Monastyrshyn and Nebehay as applied to claims 4 and 12 above, and further in view of Scavezze (U.S. Patent Application Publication No. 2013/0335405), hereinafter referenced as Scavezze.
Regarding claim 5, the combination of Monastyrshyn and Nebehay teaches wherein the video is captured by a user device, (Monastyrshyn, paragraph 57 teaches “visualization module 102c may receive motion captured by one or more camera devices 107 via camera module 102b”); this shows aforementioned video captured via user device.
However, the combination of Monastyrshyn and Nebehay fails to explicitly teach
wherein the second 3D position is closer to the user device than the first 3D position, and wherein the fourth 3D position appears closer to the user device than the third 3D position.
However, Scavezze teaches wherein the second 3D position is closer to the user device than the first 3D position, (Scavezze, paragraph 32 teaches “providing a mixed reality experience by fusing virtual content 21 (completed virtual content in this example) with real content 27 within a user's FOV.” and paragraph 124 teaches “a user moves closer to a virtual object”); as shown in fig. 1, moving closer to virtual object (from first to second 3D position) would lead user closer to actual object 27, thus the second 3D position is closer to user device 2 depicted in the fig. 1; and wherein the fourth 3D position appears closer to the user device than the third 3D position (Scavezze, paragraph 124 teaches “As a user moves around within a scene, and changes his position and/or FOV, the appearance of virtual objects will change. For example, if a user moves closer to a virtual object, the object may be projected larger”); virtual object appearing larger in the fourth/closer position means it appears closer to the user device 2 depicted in fig. 1 when compared to the previous/third 3D position. Scavezze is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of position of object and virtual object appearing closer/farther due to movement in a realistic manner. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Monastyrshyn and Nebehay with the virtual object sizing and appearance relative to a user device techniques of Scavezze to improve the ability of the user to fit the scale and perspective of virtual objects together in the environment (Scavezze, paragraph 3). This would be from the perspective affecting size and appearance aforementioned leading to a better user experience. Regarding claim 6, the combination of Monastyrshyn, Nebehay and Scavezze teaches further comprising increasing a size of the virtual item proportionally by an amount of increase in size of the object from the first frame to the second frame (Scavezze, paragraph 124 teaches “As a user moves around within a scene, and changes his position and/or FOV, the appearance of virtual objects will change. For example, if a user moves closer to a virtual object, the object may be projected larger”); moving closer leading to virtual object projected larger means increasing size of virtual item which one of ordinary skill in the art would understand must be done proportionally [so realistic ratio can be maintained and to avoid distorting] to an amount of increase in size of object (such as object 27 in fig. 1) from first to second frame. The same motivations used in claim 5 apply here in claim 6.
Regarding claim 7, the combination of Monastyrshyn, Nebehay and Scavezze teaches wherein the video is captured by a user device, (Monastyrshyn, paragraph 57 teaches “visualization module 102c may receive motion captured by one or more camera devices 107 via camera module 102b”); this shows aforementioned video captured via user device; wherein the second 3D position is farther from the user device than the first 3D position, (Scavezze, paragraph 32 teaches “providing a mixed reality experience by fusing virtual content 21 (completed virtual content in this example) with real content 27 within a user's FOV.” and paragraph 124 teaches “a user moves closer to a virtual object”); moving closer to virtual object means user can also move farther from virtual object (from first to second 3D position) [as shown in fig. 1], thus moving farther would lead user farther to actual object 27, thus the second 3D position is farther to user device 2 depicted in the fig. 1 when user moves away; and wherein the fourth 3D position appears farther from the user device than the third 3D position (Scavezze, paragraph 124 teaches “As a user moves around within a scene, and changes his position and/or FOV, the appearance of virtual objects will change. For example, if a user moves closer to a virtual object, the object may be projected larger”); since user moves around and virtual object appearing larger in the closer position means virtual object would appear smaller/farther in a farther position, thus, when user moves farther, virtual item at fourth 3D position from the user device 2 depicted in fig. 1 when compared to the previous/third 3D position appears farther away from user device. The same motivations used in claim 5 apply here in claim 7.
Regarding claim 8, the combination of Monastyrshyn, Nebehay and Scavezze teaches further comprising decreasing a size of the virtual item proportionally by an amount of increase in size of the object from the first frame to the second frame (Scavezze, paragraph 124 teaches “As a user moves around within a scene, and changes his position and/or FOV, the appearance of virtual objects will change. For example, if a user moves closer to a virtual object, the object may be projected larger”); since user can move around in a scene, using the same logic here, moving farther from virtual item would lead to virtual object projected smaller meaning decreasing size of virtual item which one of ordinary skill in the art would understand must be done proportionally [so realistic ratio can be maintained and to avoid distorting] to an amount of increase in size of object (such as object 27 in fig. 1) from first to second frame when also moving closer to object 27 simultaneously, thus, when a user is moving away from virtual item, but closer to object, this would occur. The same motivations used in claim 5 apply here in claim 8.
Regarding claim 9, the combination of Monastyrshyn, Nebehay and Scavezze teaches further comprising: determining that the object has moved left or right by a given distance (Scavezze, paragraph 97 teaches “As a user turns his head, for example left to right or up and down, the relative position of real-world objects in the user's FOV inherently moves within the user's FOV. For example, plant 27 in FIG. 1 may appear on the right side of a user's FOV at first. But if the user then turns his head toward the right, the plant 27 may eventually end up on the left side of the user's FOV”); this shows determination of object 27 moving left or right by a distance in the user’s FOV; and in response to determining that the object has moved left or right by the given distance, moving a virtual item left or right by the given distance (Scavezze, paragraph 98 teaches “user is looking at a virtual object in his FOV, if the user moves his head left to move the FOV left, the display of the virtual object may be shifted to the right by an amount of the user's FOV shift”); this shows shifting virtual object right and would be after (thus in response to) determination of object moving left or right by the aforementioned distance. The same motivations used in claim 5 apply here in claim 9.
Regarding claim 13, the combination of Monastyrshyn, Nebehay and Scavezze teaches wherein the second virtual item is placed at a specified position relative to the object and the first virtual item, (Monastyrshyn, paragraph 70 teaches “second augmented reality element may be presented within the graphical user interface spaced a distance apart from the face and the reference feature… the second augmented reality element 1010 may be initially presented in a manner similar to that of the first augmented reality element”); spaced distance apart from reference feature shows the second virtual item is positioned relative to object and since presented in manner similar to first virtual item, it is also relative to the first virtual item; further comprising receiving input that selects the reference point of the object (Scavezze, paragraph 144 teaches “The user could further perform predefined gestures to rotate the virtual object (about pitch, yaw and/or roll axes), or bend the virtual object about a selected point in the object to a desired degree (for example by placing one hand at the bending point, and pushing the object with the other hand to bend about the selected point)”); user performing operations of object at selected point indicates user input received and it must be for selecting reference point of object in order to have a selected point. The same motivations used in claim 5 apply here in claim 13.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Monastyrshyn and Nebehay as applied to claim 1 above, and further in view of Price et al. (U.S. Patent No. 10,867,447), hereinafter referenced as Price.
Regarding claim 10, the combination of Monastyrshyn and Nebehay fails to teach
further comprising: determining that the object has turned around within a video at a given rate; and in response to determining that the object has turned around within the video at the given rate, turning a virtual item around at the given rate.
However, Price teaches
further comprising: determining that the object has turned around within a video at a given rate (Price, col. 1, lines 38-40 teach “determine a position (e.g., rotation) and a scale of the physical object based at least in part on the generated digital outline”); this shows determining object has rotated (turned around) in the aforementioned video and would have to be at a given rate; and in response to determining that the object has turned around within the video at the given rate, turning a virtual item around at the given rate (Price, col. 1, lines 40-43 teach “The computing device may configure (e.g., rotate, scale) a 3D model of the physical object to match the determined position and scale of the physical object”); this shows turning around 3D model (virtual item) after (in response to) determining object has turned around and one of ordinary skill in the art would understand this would be at the same given rate for a smooth transition and consistency. Price is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of turning object and virtual object. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Monastyrshyn and Nebehay with the rotation and matching techniques of Price to ensure allowing high precision in the AR process (col. 3, line 36). This leads to better user experience and is due to the matching of 3D model to determined position of physical object.
Claim(s) 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Monastyrshyn and Nebehay as applied to claim 1 above, and further in view of DECODED (Blender Tutorial: Duplicate Objects Around A Circle (2020)) [https://www.youtube.com/watch?v=lFnUtH4SjsE], hereinafter referenced as DECODED.
Regarding claim 16, the combination of Monastyrshyn and Nebehay teaches and animating the plurality of identical virtual items (Monastyrshyn, paragraph 46 teaches “the one or more augmented reality elements comprise an interactive environment. In such embodiments, the face 402 may be presented within or on a graphically generated body, such as an animated character, an avatar, a suit (e.g., a space suit, a flight suit, a set of clothing), or any other suitable body”); this shows virtual items (augmented reality elements) would be animated.
However, the combination of Monastyrshyn and Nebehay fails to teach comprising: duplicating a virtual item to generate a plurality of identical virtual items based on a user selected duplication factor; causing the plurality of identical virtual items to be evenly distributed around the object depicted in a video; and animating the plurality of identical virtual items.
However, DECODED teaches comprising: duplicating a virtual item to generate a plurality of identical virtual items based on a user selected duplication factor (1:36-2:00 [see fig. 1 of this action] teaches creating copies of virtual item based on user selected duplication factor of 18);
PNG
media_image1.png
1017
1346
media_image1.png
Greyscale
Figure 1
causing the plurality of identical virtual items to be evenly distributed around the object depicted in a video (2:45-3:23 [see fig. 2 of this action] teaches evenly distributing (every 20 degrees) the virtual items around object); this would be done in the video from the aforementioned combination above; DECODED is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of duplicating objects in a specific manner. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Monastyrshyn and Nebehay with the duplication techniques of DECODED to quickly duplicate objects around a central point (DECODED, video description). This would lead to better user experience due to a quick duplication.
PNG
media_image2.png
991
1344
media_image2.png
Greyscale
Figure 2
Claim(s) 11 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Monastyrshyn and Nebehay as applied to claim 1 above, and further in view of Scavezze and Beaumont (U.S. Patent Application Publication No. 2004/0207651), hereinafter referenced as Beaumont.
Regarding claim 11, the combination of Monastyrshyn and Nebehay teaches wherein the object is depicted in a first frame of a video, (Monastyrshyn, fig. 4 shows object of interest 402 displayed in first frame of video 400 and paragraph 48 teaches “object of interest 402, the movement component 240 may determine a first position for the reference feature 408 in a first frame (e.g., FIG. 4)”); this shows object depicted in first frame of video.
However, the combination of Monastyrshyn and Nebehay fails to explicitly teach
further comprising: determining that a depiction of the object has failed to be detected in a second frame of the video; in response to determining that the depiction of the object has failed to be detected in the second frame of the video, presenting a list of objects detected in the second frame; receiving input that selects a given object from the list of objects; computing a reference position of the given object; and updating a 3D position of a virtual item based on the computed reference position of the given object selected by the received input from the list of objects.
However, Scavezze teaches
receiving input that selects a given object from the list of objects (Scavezze, paragraph 127 teaches “hub may include menus of predefined objects from which a user can select. A user can use the predefined objects as is, or the user can edit them as explained below. Templates of predefined objects may be provided by an author of the content-generation software application” and fig. 12 step 724 teaches user selecting object from template); menu and template of predefined objects shows list of such and user selecting object means user input received to select an object from the list; computing a reference position of the given object (Scavezze, fig. 12, step 726 teaches user interaction received for where to place object and step 730 teaches calculate position of object); this shows reference position computed of given object; and updating a 3D position of a virtual item based on the computed reference position of the given object selected by the received input from the list of objects (fig. 12, step 736 teaches prompting user to move object which leads to repeating steps 726 and 730); moving object would mean updating 3D position of virtual item (when object is virtual item) and this is based on aforementioned computed reference position of object selected by received input from aforementioned list of objects since happens in a step after. The same motivations used in claim 5 apply here in claim 11.
However, the combination of Monastyrshyn, Nebehay and Scavezze fails to teach further comprising: determining that a depiction of the object has failed to be detected in a second frame of the video; in response to determining that the depiction of the object has failed to be detected in the second frame of the video, presenting a list of objects detected in the second frame;
However, Beaumont teaches further comprising: determining that a depiction of the object has failed to be detected in a second frame of the video (Beaumont, paragraph 63 teaches “where the respective detection scheme returned any one of the object group failed detection statuses”); this shows object failed detection status which indicates determination that depiction of the object has failed to be detected and this would be second frame of video when viewed in combination with the references above since aforementioned object is determined in first frame; in response to determining that the depiction of the object has failed to be detected in the second frame of the video, presenting a list of objects detected in the second frame (Beaumont, paragraph 63 teaches “amending the object lists of existing entries to include the current object where the respective detection scheme 421 to 423 returned a `detected` detection status, and removing any object groups that fail to correctly form where the respective detection scheme returned any one of the object group failed detection statuses”); accessing list to remove object that failed detection shows that: in response to aforementioned determination (that depiction of the object has failed to be detected in the video), list of objects detected in the second frame are presented since the list must be accessed to remove object. Beaumont is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of failed detection of object and lists associated with such. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Monastyrshyn, Nebehay and Scavezze with the list and detection of object techniques of Beaumont to more efficiently convert the graphic objects to pixels (Beaumont, paragraph 39). This would mean better accuracy due to efficiently conversion of objects to pixels meaning a more detailed detection.
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Monastyrshyn, Nebehay and DECODED as applied to claim 16 above, and further in view of Moraes et al. (U.S. Patent Application Publication No. 2013/0091206), hereinafter referenced as Moraes.
Regarding claim 17, the combination of Monastyrshyn, Nebehay and DECODED fails to teach further comprising: setting a maximum limit to a number of duplicates of the virtual item based on a type associated with the virtual item, a first type of virtual items being associated with a first maximum duplication quantity and a second type of virtual items being associated with a second maximum duplication quantity.
However, Moraes teaches further comprising: setting a maximum limit to a number of duplicates of the virtual item based on a type associated with the virtual item, (Moraes, paragraph 42 teaches "identify a plurality of avatars that includes the avatar associated with the at least one contact object, calculate a number of avatars, from the plurality of avatars, needed to form a matrix that fills a display, the calculating based upon a size of the display, and determine there are fewer avatars then needed to fill the display, and duplicate the avatars until the display is filled"); this shows maximum limit (calculated amount that display fits) of virtual item duplication based on type such as avatar being set; a first type of virtual items being associated with a first maximum duplication quantity and a second type of virtual items being associated with a second maximum duplication quantity (Moraes, claim 28 teaches “a processor to calculate a number of avatars, from the plurality of avatars, needed to form a matrix that fills a display, the calculating based upon a size of the display; a first display module to determine there are more avatar then needed to fill the display; and a second display module to determine which contact objects associated with which avatars are most active and removing non-active avatars until there are a number of avatars identified to fill the display.”); one of ordinary skill in the art would understand avatar can have multiple types (associated with contact objects here and/or active/inactive) thus type 1 of avatar being associated with first maximum duplication quantity would be different from second type of virtual item (type 2 of avatar) since they would differ in shape and sizes thus taking different amounts to fill in the display, additionally, and when viewed in combination, another type would also be considered the virtual items that Moraes in claim 16 duplicates and however much of those can fit onto/fill the display.
Moraes is considered to be analogous art because it is reasonably pertinent to the problem faced by the inventor of having a maximum amount of virtual items duplicated in an efficient manner. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Monastyrshyn, Nebehay and DECODED with the maximum limit of duplicates techniques of Moraes to calculate how many contacts are needed to form a matrix filling the whole display, considering the device display size (Moraes, paragraph 22). This would lead to an exact and accurate maximum number due to calculation and lead to a more individualized and improved experience by considering the device display size.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hammack (U.S. Patent No. 3,996,590), claim 3 teaches “method for the determination of the position of a moving object… identifying each of a plurality of fixed points in space… calculating the position of said moving object relative to said identified fixed points”; this shows identifying fixed points in space (would move less relative to others) and then calculating/tracking movement of object using such.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NAUMAN U AHMAD whose telephone number is (703)756-5306. The examiner can normally be reached Monday - Friday 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/N.U.A./Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights