DETAILED ACTION
*Note in the following document:
1. Texts in italic bold format are limitations quoted either directly or conceptually from claims/descriptions disclosed in the instant application.
2. Texts in regular italic format are quoted directly from cited reference or Applicant’s arguments.
3. Texts with underlining are added by the Examiner for emphasis.
4. Texts with
5. Acronym “PHOSITA” stands for “Person Having Ordinary Skill In The Art”.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities: The disclosure recites a effect in multiple places (e.g. )[0002], [0005] etc.). Suggest replacing “a” with “an”.
Appropriate correction is required.
Claim Objections
Claims 1 and 19-20 are objected to because of the following informalities: Claims 1/19/20 recite(s) a effect prop. Suggest replacing “a” with “an”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 8-9, 13-15 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 8 recites the limitation "the image channels" in line 5. Claim 9 recites the screen of the device (last line). Claim 13 recites the camera in line 6. Claim 14 recites the augmented reality scenario, the animation enhancement effect and the visual enhancement effect. Claim 15 recites the augmented reality scenario. There is insufficient antecedent basis for these limitations in the claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 9-10, 12-14 and 19-21 are rejected under 35 U.S.C. 103 as being unpatentable over Hou (US 2021/0118237 A1) in view of Xu (CN 105892651 A).
Regarding Claim 1, Hou discloses an effect prop (Fig.5: notice the virtual object S501) display method ([0005]: The embodiments of the disclosure provide an AR scene image processing method, which may include that: shooting pose data of an AR device is acquired; presentation special effect data of a virtual object corresponding to the shooting pose data in a reality scene is acquired based on the shooting pose data and pose data of the virtual object in a three-dimensional scene model representing the reality scene; and an AR scene image is displayed through the AR device based on the presentation special effect information), comprising:
receiving a trigger operation of an effect prop in an execution device ([0078]: In some examples, updating the presentation special effect information presented in the AR scene image may refer to triggering updating of a picture of the virtual object in the AR scene, triggering updating of a sound playing effect corresponding to the virtual object, or triggering updating of smell diffusion corresponding to the virtual object, or triggering multiple combinations of updating of the picture of the virtual object in the AR scene, updating of the sound playing effect corresponding to the virtual object, or updating of smell diffusion corresponding to the virtual object), wherein the execution device currently has first pose information ([0005]: The embodiments of the disclosure provide an AR scene image processing method, which may include that: shooting pose data of an AR device is acquired; presentation special effect data of a virtual object corresponding to the shooting pose data in a reality scene is acquired based on the shooting pose data and pose data of the virtual object in a three-dimensional scene model representing the reality scene; and an AR scene image is displayed through the AR device based on the presentation special effect information);
displaying an enhancement effect of the effect prop in a set display state (Fig.5 and [0075]: FIG. 5 shows an AR scene. In FIG. 5, a virtual object S501 is a virtual dinosaur, and a reality scene S502 is a building. A building image displayed in FIG. 5 is a three-dimensional scene model corresponding to the reality scene. Under the condition of determining, based on a position coordinate of the three-dimensional scene model, shooting position data of the AR device and pose data of the virtual dinosaur in the three-dimensional scene model, that the virtual dinosaur is occluded by a physical object (i.e., the building) in the reality scene corresponding to the three-dimensional scene model, the occluded part of the virtual dinosaur will not be rendered, and the three-dimensional scene model may be in a transparent state during the rendering process. As a result, an AR user may view a realistic occlusion effect through the AR device);
PNG
media_image1.png
451
398
media_image1.png
Greyscale
receiving a pose adjustment operation on the execution device, wherein the first pose information of the execution device is changed into second pose information ([0046]: In an implementation mode, as shown in FIG. 2, in the operation that the shooting pose data is determined based on the reality scene image collected by the camera, the following operations S201 to S202 may be executed. In S201, a reality scene image shot by the AR device is acquired. In S202, shooting pose data corresponding to the reality scene image is determined based on the reality scene image and a pre-stored first neural network model for positioning, where the shooting pose data includes shooting position information and/or shooting orientation information).
Hou fails to disclose keeping displaying the enhancement effect of the effect prop based on maintaining the set display state.
However Xu, in the same field of augmented reality application, discloses the display mode of virtual objects can be controlled by setting the corresponding display parameters for different types of virtual objects ([0049]). Xu discloses The first-class virtual objects can be virtual objects with specific object attributes in the three-dimensional virtual objects. For example, the first type of virtual object can be a virtual object with dynamic attributes in a virtual scene, such as an animal, a human, etc. The second type of virtual object can be a virtual object with object attributes that are different from those of the first type of virtual object. For example, the second type of virtual object can be an object with static properties in a three-dimensional virtual scene, such as walls, trees, furniture, etc., which are usually in a relatively static state relative to the virtual scene ([0064]-[0065]). Xu further discloses wherein, in the second display mode, when the viewer of the virtual object moves, the second type of virtual object is in a static and fixed state relative to the real space and does not move with the viewer's movement ([0061]).
Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Xu into that of Hou and to include the limitation of keeping displaying the enhancement effect of the effect prop based on maintaining the set display state in order to solve the technical problem of poor flexibility in display methods when electronic devices display virtual objects as suggested by Xu ([0008]).
Regarding Claim 2, Hou discloses AR scene may be information associated with the reality scene, such as an animation, a text and a picture. ([0104]). Xu discloses for virtual object such as wall, this type of virtual object can have static characteristics ([0066]). Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Hou and Xu and to include the limitation of wherein the enhancement effect comprises a visual enhancement effect (Hou [0002]: In an AR technology, physical information (visual information, a sound, a touch and the like) is simulated and then superimposed on the real world, and thus a real environment and a virtual object are presented in the same screen or space in real time); and displaying the enhancement effect of the effect prop in the set display state comprises: playing an animation enhancement effect of the effect prop in an augmented reality scenario (Hou [0104]: In addition, embodiments of the disclosure also provide an AR scene image processing method. In the method, a preset identifier may be added to the reality scene, and the preset identifier stores preset identification information mapped with additional virtual object information. Here, the additional virtual object information may be information associated with the reality scene, such as an animation, a text and a picture), and presenting the animation enhancement effect at a set screen position of the execution device in a predetermined effect display size (Xu [0073]: Display parameters can include the display duration, position, size, and effects of virtual objects during the rendering process, in order to determine how virtual objects are displayed during the rendering process. [0075]: The second type of display parameters can include those used to prevent virtual objects from moving in a 3D scene. That is, even if the viewer moves while watching the virtual object, the virtual object remains stationary relative to the real space and does not move with the viewer. In other words, virtual objects with the second type of display parameters are stationary relative to the display scene during the display process, and they have a second display mode. [0066]: Alternatively, if the object's properties indicate that the virtual object is a tree, room, wall, etc., then this type of virtual object can have static characteristics) since it only takes a skilled person a routine skill to include animation virtual visual image on a virtual wall in order to make profit through providing animated advertisement to related manufactures or business owners.
Regarding Claim 3, Hou further teaches or suggests during the process of playing the animation enhancement effect, in a case that the animation enhancement effect is played to a preset time point, superimposing a light enhancement effect and presenting the same (Hou [0092]: In some examples, the attribute information refers to a specific type of a reality scene represented by the reality scene image, and may be represented by a tag identification. For example, the same indoor space may be decorated to multiple decoration types, and each decoration type may correspond to a presentation special effect of a virtual object. For example, the virtual object may be virtual ceiling lamps that emit light with different colors. [0104]: In the method, a preset identifier may be added to the reality scene, and the preset identifier stores preset identification information mapped with additional virtual object information. Here, the additional virtual object information may be information associated with the reality scene, such as an animation, a text and a picture. It would have been obvious to a PHOSITA before the effective filing date of the claimed invention to include an animation with different lighting configuration).
Regarding Claim 9, Hou modified by Xu further teaches or suggests wherein during the process of playing the animation enhancement effect, the method further comprises: according to the first pose information of the execution device, determining the effect display size of each effect video frame in the animation enhancement effect on the screen of the device (Hou [0005]: The embodiments of the disclosure provide an AR scene image processing method, which may include that: shooting pose data of an AR device is acquired; presentation special effect data of a virtual object corresponding to the shooting pose data in a reality scene is acquired based on the shooting pose data and pose data of the virtual object in a three-dimensional scene model representing the reality scene; and an AR scene image is displayed through the AR device based on the presentation special effect information. [0104]: Here, the additional virtual object information may be information associated with the reality scene, such as an animation, a text and a picture. Xu [0036]: Virtual objects with a second attribute parameter are identified as the second type of virtual objects. [0073]: Display parameters can include the display duration, position, size, and effects of virtual objects during the rendering process, in order to determine how virtual objects are displayed during the rendering process). The same reason to combine as that of Claim 1 is applied.
Regarding Claim 10, Hou modified by Xu further teaches or suggests according to the first pose information of the execution device, determining a space coordinate point of a camera in the augmented reality scenario; acquiring initial plane information of a preset initial vertical plane in the augmented reality scenario; and according to the space coordinate point and the initial plane information, and in combination with the set screen position, determining the effect display size of each effect video frame in the animation enhancement effect on the screen of the device (Hou [0030]: In some examples, the shooting pose data of the AR device may include a position and/or display orientation of a display component configured to display a virtual object when a user holds or wears the AR device. For conveniently explaining the shooting pose data, a concept of coordinate system, for example, world coordinate system, is introduced. The shooting pose data includes a coordinate position of the display component of the AR device in the world coordinate system, or includes an included angle between the display component of the AR device and each coordinate axis in the world coordinate system, or includes both the coordinate position of the display component of the AR device in the world coordinate system and the included angle with each coordinate axis in the world coordinate system. [0047]: In some examples, after the reality scene image collected by the camera of the AR device is acquired, the reality scene image may be input to the pre-trained first neural network model for positioning to obtain the shooting pose data corresponding to the reality scene image. Xu [0066]: if the object's properties indicate that the virtual object is a tree, room, wall, etc., then this type of virtual object can have static characteristics. [0073]: Display parameters can include the display duration, position, size, and effects of virtual objects during the rendering process, in order to determine how virtual objects are displayed during the rendering process. A PHOSITA would have known that a virtual wall is a vertical plane in the AR scenario and virtual object). The same reason to combine as that of Claim 1 is applied.
Regarding Claim 12, Hou modified by Xu teaches or suggests continuing to play the animation enhancement effect of the effect prop in the augmented reality scenario, and presenting the animation enhancement effect at the set screen position of the execution device in the effect display size (Xu [0075]: The second type of display parameters can include those used to prevent virtual objects from moving in a 3D scene. That is, even if the viewer moves while watching the virtual object, the virtual object remains stationary relative to the real space and does not move with the viewer. In other words, virtual objects with the second type of display parameters are stationary relative to the display scene during the display process, and they have a second display mode).
Regarding Claim 13, Hou modified by Xu further teaches or suggests constructing a target vertical plane in the augmented reality scenario according to the second pose information of the execution device; and controlling the camera in the augmented reality scenario to capture the animation enhancement effect, and presenting the animation enhancement effect on the target vertical plane, so that the animation enhancement effect presented at the set screen position of the execution device maintains the effect display size (Xu [0066]: if the object's properties indicate that the virtual object is a tree, room, wall, etc., then this type of virtual object can have static characteristics. Virtual wall implied a vertical plane in AR scenario. [0075]: The second type of display parameters can include those used to prevent virtual objects from moving in a 3D scene. Note Hou teaches the virtual object may be information associated with the reality scene, such as an animation, a text and a picture, see [0104]. It would only require a skilled person to have routine skill to include a virtual wall with animation images displayed on the wall in order to promote products for advertisement profits). The same reason to combine as that of Claim 1 is applied.
Regarding Claim 14, Hou teaches or suggests displaying the enhancement effect of the effect prop in the set display state comprises: in the augmented reality scenario, playing a sound enhancement effect of the effect prop at a set sound effect playing rate, wherein the sound enhancement effect is synchronously played with the animation enhancement effect of the visual enhancement effect that is comprised in the enhancement effect ([0078]: In some examples, updating the presentation special effect information presented in the AR scene image may refer to triggering updating of a picture of the virtual object in the AR scene, triggering updating of a sound playing effect corresponding to the virtual object, or triggering updating of smell diffusion corresponding to the virtual object, or triggering multiple combinations of updating of the picture of the virtual object in the AR scene, updating of the sound playing effect corresponding to the virtual object, or updating of smell diffusion corresponding to the virtual object).
Regarding Claim 19, Claim 19 is/are similar to Claim 1 except in the format of device. Hou further discloses the device comprising a processor; and a storage apparatus, configured to store a program, wherein when the program is executed by the process, the processor implements the acts defined by the method (see Fig.13/14: Processor 1101/1201 and Memory 1102/1202). Therefore the same reason(s) for rejection is/are applied to Claim 1 is/are also applied to Claim 19.
Regarding Claims 20-21, Claims 20-21 is/are similar to Claims 1-2 except in the format of non-transitory computer-readable storage medium. Therefore the same reason(s) for rejection is/are applied to Claim 1-2 is/are also applied to Claim 20-21.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Hou (US 2021/0118237 A1) in view of Xu (CN 105892651 A) as applied to Claim 2 above, and further in view of Stolzberg et al. (US 2018/0192042 A1).
Regarding Claim 4, Hou modified by Xu fails to disclose wherein playing the animation enhancement effect of the effect prop in the augmented reality scenario comprises: acquiring a video file corresponding to the effect prop, wherein the video file is stored in a set video format; and decoding the video file, and playing decoded effect video frames in the augmented reality scenario.
However Stolzberg discloses virtual video wall had already been known to be used in the field of augmented reality ([0102]: A virtual display 540 exists entirely in virtual space, such as on a website, gaming site, immersive environment, virtual reality environment, augmented reality environment, or any other virtual setting. In this regard, instead of the videowall content control application outputting content to a display, the videowall control application may output to a virtual space. For instance, the videowall engine may generate one or more virtual cameras associated with a virtual space, and map content to that virtual space through the one or more virtual cameras. As such, virtual video walls may be generated within a virtual environment. Such virtual video walls may be interactive within the virtual space. In this regard, users may access and interact with information, transactions, entertainment and other such content, on the virtual video wall similarly to the way that a user may access and interact with a physical video wall in physical space). Stolzberg further discloses a[0040] The videowall control application may pass the configuration file into a compiler which may then transform the configuration instructions into machine readable language instructions and data which are executable and/or readable by a processor. Based on the instructions, the processor may then transmit content to a videowall engine. Content may include, but is not limited to, all types of multimedia including video files, live action recordings, two and/or three dimensional virtual reality content, audio content, 3D objects, etc. The processor may also instruct the videowall engine how the particular content is to be mapped to a video wall. In this regard, the processor may provide the videowall engine with video wall data regarding the number of displays in the video wall, the resolution of the displays, and the locations of the displays relative to one another ([0040]) and Media transceivers may receive and decode the transmitted content and provide the content for display on the video wall ([9942]).
Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Stolzberg into that of Hou modified by Xu and to include the limitation of wherein playing the animation enhancement effect of the effect prop in the augmented reality scenario comprises: acquiring a video file corresponding to the effect prop, wherein the video file is stored in a set video format; and decoding the video file, and playing decoded effect video frames in the augmented reality scenario in order to meet the need for a platform that uses multiple displays to deliver visual information and uses that display real estate (that is, the area on the displays available for presenting visual information) to present the information in intelligent ways, thereby increasing its value as suggested by Stolzberg ([0004]).
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Hou (US 2021/0118237 A1) in view of Xu (CN 105892651 A) and Stolzberg et al. (US 2018/0192042A1) as applied to Claim 4 above, and further in view of Xiang et al. (US 2019/0289311 A1).
Regarding Claim 5, Hou as modified fails to explicitly recite wherein decoding the video file, and playing the decoded effect video frames in the augmented reality scenario comprises: acquiring playing parameters and the current playing information of the video file; decoding the next effect video frame to be played according to the playing parameters and the current playing information, and playing the next effect video frame to be played in the augmented reality scenario.
However acquiring playing parameters and the current playing information of the video file; decoding the next effect video frame to be played according to the playing parameters and the current playing information, and playing the next effect video frame to be played in the augmented reality scenario are conventional technical means in the video playing art. Without knowing playing parameters and current playing information, it would have been impossible for the video to be displayed and without decoding the next video frame the stream video could be continuously played without glitch. Xiang discloses acquiring playing parameters and the current playing information of the video file ([0079]: Video file analysis unit 402 may be configured to determine and/or analyze a parameter of the video file. The parameter of the video file may include a type of a frame, the lengths of the frames of the video file, the locations of the frames of the video file, the number of the frames in the video file, a frame index of the video file, the size of the video file, or the like, or a combination thereof); decoding the next effect video frame to be played according to the playing parameters and the current playing information, and playing the next effect video frame to be played in the augmented reality scenario ([0012]: In some embodiments, the data of the video fragment may be decoded. A moving object may be detected based on the decoded video fragment. The moving object information may be extracted based on the moving object from the decoded video fragment. [0085]: Decoding unit 502 may decode the video fragments based on a decoding method. For example, decoding unit 502 may decompress the video fragment for further processing. [0139]: In some embodiments, if the time duration of the video file is relatively long, for example, 24 hours, background images of some of the video fragments may be an image relating to daytime, and otherwise background images of some of the video fragments may be an image relating to night time. If video synopsis generation module 308 selects a background image relating to daytime as the background image of the video synopsis, the display effect of the moving object(s) that occur during the night time may be poor. Therefore, in some embodiments, video synopsis generation module 308 may generate the video synopsis by grouping the video fragments. For example, for a video fragment with a time duration of 3 minutes, video synopsis generation module 308 may select 5 video fragments as a group and select a background image for the 5 video fragments, and select another background image for next 5 video fragments, and so forth. Accordingly, the time duration of the video synopsis may be relatively long, but the display effect may be good).
Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Xiang into that of Hou as modified and to include the limitation of acquiring playing parameters and the current playing information of the video file; decoding the next effect video frame to be played according to the playing parameters and the current playing information, and playing the next effect video frame to be played in the augmented reality scenario in order to play video according to the video file.
Regarding Claim 6, Xiang further teaches or suggests wherein decoding the next effect video frame to be played according to the playing parameters and the current playing information comprises: determining the next video frame index according to the playing parameters and the current playing information of the video file, and in combination with a set playing mode; and determining the next effect video frame to be played from the video file according to the next video frame index ([0068]: In some embodiments, video segmentation module 302 may segment the video file based on a parameter of the video file. For example, for a non-stream video file, video segmentation module 302 may segment the video file based on a frame index of the video file. [0079]: Video file analysis unit 402 may be configured to determine and/or analyze a parameter of the video file. The parameter of the video file may include a type of a frame, the lengths of the frames of the video file, the locations of the frames of the video file, the number of the frames in the video file, a frame index of the video file, the size of the video file, or the like, or a combination thereof. For example, video file analysis unit 402 may generate a frame index based on the type of frame, the length of the frame, the location of the frame, and the number of frames in the video file. In some embodiments, the video file may include a stream video file or a non-stream video file. In some embodiments, video file analysis unit 402 may transcode the non-stream video file to a stream video file). The same reason to combine as that of Claim 5 is applied.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Hou (US 2021/0118237 A1) in view of Xu (CN 105892651 A) and Stolzberg et al. (US 2018/0192042A1) and Xiang et al. (US 2019/0289311 A1) as applied to Claim 6 above, and further in view of Attorre et al. (US 2019/0035431 A1).
Regarding Claim 7, Xiang further teaches In some embodiments, a parameter of the video file may be analyzed. A computing power may be determined. A parameter of a video segment may be determined based on the parameter of video file and the computer power. The plurality of video fragments may be generated based on the parameter of the video file and the parameter of the video segment. In some embodiments, the parameter of the video file may include a type of a frame, a length of the frame, a location of the frame, the number of the frames in the video file, a frame index of the video file, or size of the video file ([0007]-[0008]). Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Xiang into that of Hou as modified and to include the limitation of extracting, from the playing parameter information, a playing duration of the video file, in order to play the virtual video on virtual video wall.
Hou as modified fails to explicitly recite extracting a playing rate of the video file.
However a PHOSITA before the effective filing date of the claimed invention would have known to obtain a playing rate before playing a video in order to play the video as intended by the video creator as Attorre ([0003]: Digital media content, such as videos and audio recordings, generally includes media content that lasts a period of time when being played. For example, a video may include a sequence of ordered image frames that can be played at a certain frame rate (e.g., 30 frames per second) and an audio recording (e.g., an audio track) that may be played simultaneously with the image frames). Attorre further discloses a PHOSITA had known that video frame index reflects beginning and end time of each video shot ([0231]: the beginning and end frame index (or time) of each shot, and metadata captions)). Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Attorre into that of Hou as modified and to include the limitation of extracting, from the playing parameter information, a playing duration of the video file, a playing rate of the video file and the number of effect video frames, and acquiring the current playing time point from the current playing information; determining a frame playing time of the video file according to the playing duration and the playing rate; and determining the next video frame index on the basis of the number of frames, the current playing time point and the frame playing time, and in combination with a frame index computation formula corresponding to the playing mode in order to play a video stream properly.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Hou (US 2021/0118237 A1) in view of Xu (CN 105892651 A) and Stolzberg et al. (US 2018/0192042A1) and Xiang et al. (US 2019/0289311 A1) as applied to Claim 6 above, and further in view of Qi et al. (CN 111193876 A).
Regarding Claim 8, Xiang teaches or suggests determining the next video frame playing position from the video file on the basis of the next video frame index ([0079]: Video file analysis unit 402 may be configured to determine and/or analyze a parameter of the video file. The parameter of the video file may include a type of a frame, the lengths of the frames of the video file, the locations of the frames of the video file, the number of the frames in the video file, a frame index of the video file, the size of the video file, or the like, or a combination thereof. For example, video file analysis unit 402 may generate a frame index based on the type of frame, the length of the frame, the location of the frame, and the number of frames in the video file. In some embodiments, the video file may include a stream video file or a non-stream video file. In some embodiments, video file analysis unit 402 may transcode the non-stream video file to a stream video file).
Hou as modified fails to disclose extracting data information of all the image channels contained in the next video frame playing position; and performing data mixing on the data information of all the image channels, and using an obtained texture image as the next effect video frame to be played in the video file.
However Qi teaches before the effective filing date of the claimed invention, a PHOSITA had already known extracting data information of all the image channels contained in the next video frame playing position; and performing data mixing on the data information of all the image channels, and using an obtained texture image as the next effect video frame to be played in the video file ([0073]: Server 200-1 is used to obtain a video file from a request and, based on the request, obtain an animation file from server 200-2 to add at least one special effect to the video file; decode the video file to obtain frame data of multiple video frames, and decode the animation file to obtain frame data of multiple animation frames, with a one-to-one correspondence between video frames and animation frames; simulate a graphics processor and run the simulated graphics processor; render the animation using the simulated graphics processor according to the frame data of each video frame and the corresponding animation frame to obtain multiple target video frames; synthesize the video based on the multiple target video frames to obtain a target video file with at least one special effect added; and send the target video file to terminal 400. Also see Fig.5/6: notice mixing texture and image) in order to add special effects to videos ([0002]). Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Qi into that of Hou as modified and to include the limitation of extracting data information of all the image channels contained in the next video frame playing position; and performing data mixing on the data information of all the image channels, and using an obtained texture image as the next effect video frame to be played in the video file in order to add special effect to the virtual visual to augmented reality applications.
Claims 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Hou (US 2021/0118237 A1) in view of Xu (CN 105892651 A) as applied to Claim 2 above, and further in view of Bloebaum (US 2009/0251421 A1).
Regarding Claim 15, Hou modified by Xu fails to disclose including a tactile enhancement effect.
However including tactile enhance effect had been a known technic in the field of in argument reality application. Bloebaum discloses a method and apparatus for augmenting visual perception of a digital image that enables a user to "feel" remote objects depicted in a visual image. Exemplary embodiments of the invention detect image texture in a digital image and generate tactile feedback control signals as a function of the detected image texture. A tactile feedback device, such as a vibrator, converts the tactile feedback control signals into tactile sensations. The vibrator may vary the intensity, frequency, and/or duty cycle of the vibration responsive to the tactile feedback control signals. In one exemplary embodiment, edge detection techniques are used to detect discontinuities in the digital image, such as sharp changes in image luminous intensity. Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Bloebaum into that of Hou as modified and to include the limitation of wherein the enhancement effect comprises a tactile enhancement effect; and displaying the enhancement effect of the effect prop in the set display state comprises: in the augmented reality scenario, in a case that a vibration enhancement condition of the effect prop is met, presenting a vibration enhancement effect by means of controlling a vibration apparatus on the execution device in order to enable a user to “feel” remote objects depicted in a visual image as suggested by Bloebaum.
Regarding Claim 16, Bloebaum teaches A tactile feedback device, such as a vibrator, converts the tactile feedback control signals into tactile sensations. The vibrator may vary the intensity, frequency, and/or duty cycle of the vibration responsive to the tactile feedback control signals. In one exemplary embodiment, edge detection techniques are used to detect discontinuities in the digital image, such as sharp changes in image luminous intensity ([0004]). Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Bloebaum into that of Hou modified by Xu and to include the limitation of acquiring vibration parameter information corresponding to the currently satisfied vibration enhancement condition; and on the basis of the vibration parameter information, controlling the vibration apparatus to vibrate to present the vibration enhancement effect, wherein the vibration parameter information comprises a vibration amplitude, a vibration frequency, and a vibration duration. The same reason to combine as that of Claim 15 is applied.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Hou (US 2021/0118237 A1) in view of Xu (CN 105892651 A) as applied to Claim 2 above, and further in view of Mitchell et al. (US 2014/0267404 A1).
Regarding Claim 17, Hou discloses In some examples, updating the presentation special effect information presented in the AR scene image may refer to triggering updating of a picture of the virtual object in the AR scene, triggering updating of a sound playing effect corresponding to the virtual object, or triggering updating of smell diffusion corresponding to the virtual object, or triggering multiple combinations of updating of the picture of the virtual object in the AR scene, updating of the sound playing effect corresponding to the virtual object, or updating of smell diffusion corresponding to the virtual object ([0078]).
Hou modified by Xu fails to disclose wherein the effect prop is a firework prop; and the enhancement effect corresponding to the firework prop comprises: a firework blooming animation, firework explosion sound, firework explosion flash, and a firework explosion vibration sense.
However including a firework in AR application had been known to a PHOSITA before the effective filing date of the claimed invention. Mitchell discloses an augmented reality device is able identify a predefined object such as an AR toy and Upon identifying the augmented reality toy, the augmented reality component 110 augmented the appearance of the augmented reality toy within the visual scene displayed on the augmented reality device (block 420). For instance, the augmented reality component 110 could determine one or more augmentations associated with the determined object type of the augmented reality toy, and could render one or more frames depicting the determined augmentation(s) applied to the captured visual scene. As an example, where the augmented reality toy is a toy castle, the augmented reality component 110 could determine that this toy is associated with a fireworks augmentation. The augmented reality component 110 could then render frames depicting virtual fireworks going off above the toy castle. Additionally, the rendered frames may augment the appearance of the toy castle as well based on the applied augmentations. For instance, the toy castle's appearance could be augmented so that it appears light from the virtual fireworks is reflecting on to the castle displayed on the augmented reality device 100. Once augmented reality component 110 renders frames depicting one or more augmentations to the visual scene, the frames are output for display (block 425) and the method 400 ends ([0042]). Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Mitchell into that of Hou modified by Xu and to include the limitation of wherein the effect prop is a firework prop; and the enhancement effect corresponding to the firework prop comprises: a firework blooming animation, firework explosion sound, firework explosion flash, and a firework explosion vibration sense in order to enhance the excitement feeling seen by the AR device user.
Allowable Subject Matter
Claim 11 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Prior art, either individually or in combination, fails to disclose or render obviousness the limitation of wherein according to the space coordinate point and the initial plane information, and in combination with the set screen position, determining the effect display size of each effect video frame in the animation enhancement effect on the screen of the device, comprises: acquiring center point coordinates of the set screen position, and determining corresponding plane point coordinates of the center point coordinates on the initial vertical plane according to the space coordinate point and the initial plane information; for each effect video frame in the animation enhancement effect, determining, on the initial vertical plane by using the plane point coordinates as the picture center point coordinates of the effect video frame, pixel point coordinates of pixel points in the effect video frame, which are presented on the initial vertical plane; and on the basis of the pixel point coordinates, determining the corresponding effect display size of the effect video frame on the screen of the device as claimed in dependent Claim 11.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YINGCHUN HE whose telephone number is (571)270-7218. The examiner can normally be reached M-F 8:00-5:00 MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao M Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YINGCHUN HE/Primary Examiner, Art Unit 2613