Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-18, 20-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation “at least one first present element” in line 4-5 and "a first present element" in 7. It is unclear if “at least one first present element” and “a first present element” refers to “a plurality of first present elements” or something else.
Claim 1 recites the limitation “continuing to dynamically present, according to the preset present mode, a first present element having relative present information with the main element being a non-persistence presentation” It is unclear how to present “a first present element having relative present information with the main element being a non-persistence presentation” without step of determining.
Claims 2-18 are rejected based on the rejection of claim 1.
Claim 8 depends from claim 1 and recites the limitation “before stacking” in line 3. The step of “stacking” is not defined in claim 1.
Claim 20 recites the limitation “at least one first present element” in lines 7-8 and "a first present element" in 11. It is unclear if “at least one first present element” and “ a first present element” refers to “ a plurality of first present elements” or something else.
Claim 20 recites the limitation “continue to dynamically present, according to the preset present mode, a first present element having relative present information with the main element being a non-persistence presentation” It is unclear how to present “a first present element having relative present information with the main element being a non-persistence presentation” without step of determining.
Claim 21 recites the limitation “at least one first present element” in lines 6-7 "a first present element" in 9. It is unclear if at least one first present element” and “a first present element” refers to “ a plurality of first present elements” or something else.
Claim 21 recites the limitation “continue to dynamically present, according to the preset present mode, a first present element having relative present information with the main element being a non-persistence presentation” It is unclear how to present “a first present element having relative present information with the main element being a non-persistence presentation” without step of determining.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
1. Claims 1-3, 12-13, 20-21 are rejected under 35 U.S.C. 103 as being unpatentable over Simon et al, IDS, CN109241465(English translated) (“Simon”) in view of LIN et al, U.S Patent Application Publication No.2021/0366163 (“LIN”)
Regarding independent claim 1, Simon teaches an image present method ([0013] The method provided in this embodiment displays moving weather elements on the interface. If the weather element collides with the target area of the interface during its movement, a deformed image of the weather element will be displayed around the target area, thereby showing the change in the shape of the weather element after it collides with the target area.”), comprising:
dynamically presenting a plurality of first present elements (see at least [0016] “Based on this implementation, the beneficial effects include at least the following: when a weather element collides with the temperature text in the interface, a deformed image of the weather element can be displayed around the temperature text, thus showing that the temperature text has responded to the weather element, presenting a dynamic effect of interaction between the weather element and the temperature text, improving the correlation between the weather element and the temperature text, and increasing the sense of immersion.”; [0164] 204. The terminal displays moving weather elements on the interface based on weather information. [0165] Weather element: A type of interface element that indicates the weather information. Weather elements can be displayed as a small image in the interface, which corresponds to the components of weather. For example, weather elements may include rain line elements, hail elements, fog elements, haze elements, snowflake elements, sand elements, dust elements, sunshine elements, cloudy elements, etc. Taking the rain line element as an example, the rain line element can indicate rainy weather. The rain line element can be displayed as a rain line image in the interface, and the rain line image can be a transparent line with a certain length and width. Taking the snowflake element as an example, the snowflake element can indicate a snowy day. The snowflake element can be displayed as a snowflake image in the interface, such as a white hexagon”) according to a preset present mode (see at least [0204] “Regarding the timing for triggering the display of the distorted image of a weather element, the distorted image of the weather element can be displayed when the display position of the weather element is close enough to the target area. In one possible implementation, a preset condition can be set for the distance between the display position of the weather element and the target area. Based on the display position of the weather element and the display position of the target area, it can be determined whether the distance between the display position of the weather element and the target area meets the preset condition. When the distance between the display position of the weather element and the target area meets the preset condition, it is confirmed that the display position of the weather element and the target area are close enough, and then the deformed image of the weather element will be displayed within the preset range of the target area. When the distance between the display position of the weather element and the target area does not meet the preset condition, it is confirmed that the display position of the weather element and the target area are far apart, and then the deformed image of the weather element does not need to be displayed temporarily.”);;
displaying, in response to determining that relative present information between at least one first present element and a main element is a persistence presentation, the at least one first present element on the main element ([0200] 206. When the distance between the display position of the weather element and the target area meets the preset conditions, the terminal displays the deformed image of the weather element within the preset range of the target area. [0201] Deformed images of weather elements are used to represent the changes in the shape of a weather element after it collides with a target area. These changes can include changes in shape, size, etc., and the deformed images of weather elements can include deformed weather elements. For example, if the weather element is a rain line element, the deformed image of the rain line element can include raindrops; if the weather element is a snowflake element, the deformed image of the snowflake element can include snow particles.”; [0209] For example, in the properties of the snowflake element's deformed image, the shape type can be a perfect circle, the color can be white, and the radius can be small. So, during the process of moving and displaying the snowflake element, when the snowflake element collides with the edge of the target area, a white circle with a very small radius can be drawn according to the properties of the rain line element's deformed image, and this white circle with a very small radius can be displayed around the target area. The small white circle can be seen as a snow particle formed by the collision of snowflake elements. [0210] In this embodiment, the deformed image of weather elements can be displayed statically in the interface. For example, the deformed weather elements can be displayed to remain on the target area. Taking snowflake elements as an example, snow particles can be displayed to remain on the target area and be deposited on the target area. It can also display deformed images of weather elements moving in the interface. For example, it can display deformed weather elements bouncing on the target area. Taking rain line elements as an example, it can display raindrops bouncing on the target area.”); and
continuing to dynamically present, according to the preset present mode, a first present element having relative present information with the main element being a non-persistence presentation (see at least [0216] The deformed weather element can be located on the upper surface of the target area, or it can be located above the target area but at a certain distance from the upper surface of the target area. With the feature (1.1), during the subsequent continuous display of multiple frames of the first image, it is possible to present the effect of deformed weather elements always bouncing above the target area, such as bouncing from the upper surface of the target area to the top of the target area, and then bouncing from the top of the target area to the upper surface of the target area, thereby simulating the phenomenon in the real world that when an object bounces, it will generally bounce above the plane due to the influence of gravity.”) Simon is understood to be silent on the remaining limitations of claim 1.
In the same field of endeavor, LIN teaches presenting, in response to an effect trigger operation, a plurality of first present elements ([0068] In step S31, a trigger condition of the special effect is set in response to a received trigger condition setting command. [0069] In the embodiment, a special effect is displayed only when a trigger condition is met, where an action, an expression, or a sound of the user, a terminal parameter, or the like may be set as the trigger condition. The action may be a facial action, such as blinking, opening the mouth, shaking the head, nodding, and raising eyebrows. For example, if the special effect is a two-dimensional sticker of glasses, the trigger condition may be set as quickly blinking twice. When detecting that the user quickly blinks twice, the two-dimensional sticker of glasses is displayed on the user's eyes. The expression may be a happy expression, a depressed expression, an angry expression, or the like. For example, if the special effect is a two-dimensional sticker of tears, the trigger condition may be set as a depressed expression. When detecting that the user has the depressed expression, the two-dimensional sticker of tears is displayed below the user's eyes. If a sound is set as the trigger condition, a voice of the user or an environmental sound may be detected, and when a predetermined sound is detected, a corresponding special effect is triggered. If a terminal parameter is set as the trigger condition, a parameter of each component in the terminal, such as a gesture or a shaking action of the terminal, may be detected, and a corresponding special effect is triggered by the gesture or the shaking action. The trigger conditions are not listed in detail herein. It should be understood that the trigger condition may be any trigger condition that is applicable to the technical solution of the present disclosure, and the number of the trigger condition may be one or more, which is not limited herein. The trigger condition may be used to trigger starting or stopping a special effect. The triggering starting a special effect means that a corresponding special effect appears when the trigger condition is met. The triggering stopping a special effect means that a corresponding special effect disappears when the trigger condition is met. The trigger condition may further include a delay time period from triggering, which means a time period after which a special effect appears or disappears. [0070] In the embodiment, the parameter of the special effect further includes a trigger condition of the special effect.”)
Therefore, it would have been obvious to one of ordinary skill in the art before effective filling date of the claimed invention to modify the method of displaying moving weather elements on the interface of Simon with including an effect trigger operation as seen in LIN because this modification would displaying the special effect only when a trigger condition is met ( [0069] of LIN)
Thus, the combination of Simon and LIN teaches an image present method, comprising: dynamically presenting, in response to an effect trigger operation, a plurality of first present elements according to a preset present mode; displaying, in response to determining that relative present information between at least one first present element and a main element is a persistence presentation, the at least one first present element on the main element; and continuing to dynamically present, according to the preset present mode, a first present element having relative present information with the main element being a non-persistence presentation.
Regarding claim 2, Simon and LIN teach the method according to claim 1, wherein the effect trigger operation comprises at least one of the following: triggering an effect display control; triggering an effect for displaying the first present elements; and detecting that a framing image comprises the main element (see at least of LIN: [0068] In step S31, a trigger condition of the special effect is set in response to a received trigger condition setting command. [0069] In the embodiment, a special effect is displayed only when a trigger condition is met, where an action, an expression, or a sound of the user, a terminal parameter, or the like may be set as the trigger condition. The action may be a facial action, such as blinking, opening the mouth, shaking the head, nodding, and raising eyebrows. For example, if the special effect is a two-dimensional sticker of glasses, the trigger condition may be set as quickly blinking twice. When detecting that the user quickly blinks twice, the two-dimensional sticker of glasses is displayed on the user's eyes. The expression may be a happy expression, a depressed expression, an angry expression, or the like. For example, if the special effect is a two-dimensional sticker of tears, the trigger condition may be set as a depressed expression. When detecting that the user has the depressed expression, the two-dimensional sticker of tears is displayed below the user's eyes. If a sound is set as the trigger condition, a voice of the user or an environmental sound may be detected, and when a predetermined sound is detected, a corresponding special effect is triggered. If a terminal parameter is set as the trigger condition, a parameter of each component in the terminal, such as a gesture or a shaking action of the terminal, may be detected, and a corresponding special effect is triggered by the gesture or the shaking action. The trigger conditions are not listed in detail herein. It should be understood that the trigger condition may be any trigger condition that is applicable to the technical solution of the present disclosure, and the number of the trigger condition may be one or more, which is not limited herein. The trigger condition may be used to trigger starting or stopping a special effect. The triggering starting a special effect means that a corresponding special effect appears when the trigger condition is met. The triggering stopping a special effect means that a corresponding special effect disappears when the trigger condition is met. The trigger condition may further include a delay time period from triggering, which means a time period after which a special effect appears or disappears. [0070] In the embodiment, the parameter of the special effect further includes a trigger condition of the special effect.”) In addition, the same motivation is used as the rejection for claim 1.
Regarding claim 3, Simon and LIN teach the method according to claim 1, further comprising: determining the preset present mode; wherein determining the preset present mode comprises: determining the preset present mode based on at least two of a preset initial position, initial velocity, and gravity information of each first present element (see at least Simon [0180] The attributes of weather elements can include color, transparency, length, width, z-coordinate, initial position coordinates (x, y), initial velocity, acceleration, initial angle, decay factor, etc. The length, width, and z-coordinate determine the size of the weather element displayed on the interface, while the transparency determines the clarity of the weather element displayed on the interface. In addition, the transparency, also known as the life value, determines the lifespan of the weather element. When the transparency of a weather element is 0, the weather element will be destroyed. The initial position determines the location of the weather element when it is first displayed. The initial position of each weather element determines the order in which they are displayed. Speed, acceleration, and decay factor determine how fast the weather element moves on the screen. The initial angle determines the direction of movement of the weather element on the screen.”; [0183] Optionally, the length, width, initial velocity, and acceleration of a weather element can be positively correlated with the intensity level of the weather information carried by the weather information; that is, the higher the intensity level of the weather, the greater the length, width, initial velocity, and acceleration of the weather element. In one possible implementation, a mapping relationship between intensity level, length of weather element, width of weather element, initial velocity of weather element, and acceleration of weather element can be established in advance. Based on weather information, the intensity level of weather can be obtained, and the mapping relationship can be queried based on the intensity level to obtain the length, width, initial velocity, and acceleration of weather element. [0184] By obtaining the length, width, initial velocity, and acceleration of weather elements based on the intensity level of the weather, it can be ensured that the size and movement speed of weather elements in the interface conform to the intensity of the weather. In an exemplary scenario, when the weather is heavy rain, the moving rain line elements on the interface are larger, wider, and move faster; when the weather is light rain, the moving rain line elements on the interface are smaller, narrower, and move slower. [0204] “Regarding the timing for triggering the display of the distorted image of a weather element, the distorted image of the weather element can be displayed when the display position of the weather element is close enough to the target area. In one possible implementation, a preset condition can be set for the distance between the display position of the weather element and the target area. Based on the display position of the weather element and the display position of the target area, it can be determined whether the distance between the display position of the weather element and the target area meets the preset condition. When the distance between the display position of the weather element and the target area meets the preset condition, it is confirmed that the display position of the weather element and the target area are close enough, and then the deformed image of the weather element will be displayed within the preset range of the target area. When the distance between the display position of the weather element and the target area does not meet the preset condition, it is confirmed that the display position of the weather element and the target area are far apart, and then the deformed image of the weather element does not need to be displayed temporarily [0216] The deformed weather element can be located on the upper surface of the target area, or it can be located above the target area but at a certain distance from the upper surface of the target area. With the feature (1.1), during the subsequent continuous display of multiple frames of the first image, it is possible to present the effect of deformed weather elements always bouncing above the target area, such as bouncing from the upper surface of the target area to the top of the target area, and then bouncing from the top of the target area to the upper surface of the target area, thereby simulating the phenomenon in the real world that when an object bounces, it will generally bounce above the plane due to the influence of gravity.”)
Regarding claim 12, Simon and LIN teach the method according to claim 1,wherein the first present element comprises at least one of a snowflake element, a water droplet element, and a hail element ([0165] of Simon “Weather element: A type of interface element that indicates the weather information. Weather elements can be displayed as a small image in the interface, which corresponds to the components of weather. For example, weather elements may include rain line elements, hail elements, fog elements, haze elements, snowflake elements, sand elements, dust elements, sunshine elements, cloudy elements, etc. Taking the rain line element as an example, the rain line element can indicate rainy weather. The rain line element can be displayed as a rain line image in the interface, and the rain line image can be a transparent line with a certain length and width. Taking the snowflake element as an example, the snowflake element can indicate a snowy day. The snowflake element can be displayed as a snowflake image in the interface, such as a white hexagon.”)
Regarding claim 13, Simon and LIN teach the method according to claim 1, further comprising: determining at least one first present element in the persistence presentation as at least one first present element to be processed (see at least Simon [0200] 206. When the distance between the display position of the weather element and the target area meets the preset conditions, the terminal displays the deformed image of the weather element within the preset range of the target area. [0201] Deformed images of weather elements are used to represent the changes in the shape of a weather element after it collides with a target area. These changes can include changes in shape, size, etc., and the deformed images of weather elements can include deformed weather elements. For example, if the weather element is a rain line element, the deformed image of the rain line element can include raindrops; if the weather element is a snowflake element, the deformed image of the snowflake element can include snow particles.”; [0209] For example, in the properties of the snowflake element's deformed image, the shape type can be a perfect circle, the color can be white, and the radius can be small. So, during the process of moving and displaying the snowflake element, when the snowflake element collides with the edge of the target area, a white circle with a very small radius can be drawn according to the properties of the rain line element's deformed image, and this white circle with a very small radius can be displayed around the target area. The small white circle can be seen as a snow particle formed by the collision of snowflake elements. [0210] In this embodiment, the deformed image of weather elements can be displayed statically in the interface. For example, the deformed weather elements can be displayed to remain on the target area. Taking snowflake elements as an example, snow particles can be displayed to remain on the target area and be deposited on the target area. It can also display deformed images of weather elements moving in the interface. For example, it can display deformed weather elements bouncing on the target area. Taking rain line elements as an example, it can display raindrops bouncing on the target area.”); and determining target velocity information of each first present element to be processed after colliding with the main element (see at least [0223] of Simon “Bouncing motion parameters can be preset to indicate the bouncing motion trajectory. These parameters may include one or more of the following: velocity, acceleration, starting direction, and decay factor. For any two adjacent first images in the first animation, the displacement of the weather element during the display of the two first images can be obtained based on the bouncing motion parameters and the time interval between displaying these two first images. Based on the display position of the weather element in the previous first image and the displacement, the display position of the weather element in the next first image can be obtained. And so on, the display position of the weather element in each first image can be obtained based on the bouncing motion parameters and the display position of the weather element in the previous first image. By obtaining the display position of the weather element in this way, and drawing the weather element at the display position obtained in the first image of each frame, it can be ensured that the first animation has the above-mentioned characteristics (2), that is, the position change of the weather element in any two adjacent first images conforms to the bouncing motion trajectory.”) , such that display information for each first present element to be processed in a next video frame is determined based on the target velocity information, and relative present information between each first present element to be processed and the main element is determined based on the display information, wherein the display information comprises display position information (see at least [0227] of Simon“For example, assuming the time interval between displaying two frames of the first image is t, and the collision position between the weather element and the target area is l1, when drawing the first frame of the first image, the weather element can be drawn at l1 of the first frame of the first image. When drawing the first image of the second frame, the displacement s1 of the weather element during the period from the first image of the first frame to the first image of the second frame can be obtained based on the speed, acceleration and t of the weather element. Based on the displacement s1 and l1, the display position l2 of the weather element in the first image of the second frame can be obtained, and the weather element can be drawn at l2 of the first image of the second frame. When drawing the first image of the third frame, the displacement s2 of the weather element during the period from the first image of the second frame to the first image of the third frame can be obtained based on the speed, acceleration and t of the weather element. Based on the displacement s2 and l2, the display position l3 of the weather element in the first image of the third frame can be obtained. The weather element is drawn at l3 in the first image of the third frame, and so on.”; [0230] For example, please refer to Figure 15. Suppose that the rain line element in the interface collides with point O on the target area. When the first animation is displayed, raindrops P will be generated at point O. Each raindrop P bounces according to a bouncing trajectory determined by different initial direction, speed, acceleration, attenuation factor and other attribute parameters. When a certain number of raindrops P are randomly generated at point O at the same time and move independently, the raindrop splashing effect is generated. In Figure 15, the angle value of angle AOB determines the range of the initial direction of the raindrop's bouncing motion, and the angle value of angle AOB can be a preset value.”; [0065] of LIN “In an embodiment, the standard face image is divided into multiple regions, such as an eye region, a nose region, a mouth region, a cheek region, an eyebrow region, and a forehead region. Each of the regions includes an optimized feature point, which is a more representative feature point selected by data analysis. The feature point represents the region in which the feature point is located. For example, if a feature point in the eye region is selected as a reference point, this indicates that the eye region is determined as a target region for forming a special effect. Multiple sub special effects may be formed for each region, and each sub special effect separately tracks the region in which it is located. The sub special effects are combined together to form a special effect. The advantage of the above operation is that the number of feature points is reduced, and it is unnecessary to select one feature point from multiple feature points as a reference point. In addition, all the feature points displayed to the user are optimized, and once the user selects a region, a feature point in the region is also selected. A large special effect may be split into multiple sub special effects, to reduce the difficulty in forming the special effect.”) In addition ,the same motivation is used as the rejection for claim 1.
Regarding independent claim 20, Simon teaches an electronic device, wherein the electronic device comprises: a processor; and a storage means, configured to store a program, and when the program is executed by the processor (see at least [0082] Thirdly, a terminal is provided, the terminal including a processor and a memory, the memory storing at least one instruction, the instruction being loaded and executed by the processor to implement the interface display method in the first aspect or any possible implementation of the first aspect. [0298] The above templates can be implemented based on hardware consisting of a processor and memory. In other words, the corresponding software can be executed by the processor running the software code stored in the memory. Specifically, each of the above modules can be a software module that performs the corresponding function within the software.”), the processor is caused to: Remaining limitations of claim 20 is similar scope to claim 1 , and therefore rejected under the same rationale.
Regarding independent claim 21, Simon teaches a non-transitory storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor ([0083] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the interface display method in the first aspect or any possible implementation thereof.”; [0124] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to perform the interface display method in the following embodiments.”), are configured to: Remaining limitations of claim 21 is similar scope to claim 1 , and therefore rejected under the same rationale.
2. Claims 4-5, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Simon et al, IDS, CN109241465 (English translated) (“Simon”) in view of LIN et al, U.S Patent Application Publication No.2021/0366163 (“LIN”) further in view of Fujioka et al., U.S Patent Application Publication No. 2010/0281408 (“Fujioka”)
Regarding claim 4, Simon and LIN teach the method according to claim 1, further comprising: before displaying, in response to determining that the relative present information between the at least one first present element and the main element is the persistence presentation, the at least one first present element on the main element ([0200] 206. When the distance between the display position of the weather element and the target area meets the preset conditions, the terminal displays the deformed image of the weather element within the preset range of the target area. [0201] Deformed images of weather elements are used to represent the changes in the shape of a weather element after it collides with a target area. These changes can include changes in shape, size, etc., and the deformed images of weather elements can include deformed weather elements. For example, if the weather element is a rain line element, the deformed image of the rain line element can include raindrops; if the weather element is a snowflake element, the deformed image of the snowflake element can include snow particles.”; [0209] For example, in the properties of the snowflake element's deformed image, the shape type can be a perfect circle, the color can be white, and the radius can be small. So, during the process of moving and displaying the snowflake element, when the snowflake element collides with the edge of the target area, a white circle with a very small radius can be drawn according to the properties of the rain line element's deformed image, and this white circle with a very small radius can be displayed around the target area. The small white circle can be seen as a snow particle formed by the collision of snowflake elements. [0210] In this embodiment, the deformed image of weather elements can be displayed statically in the interface. For example, the deformed weather elements can be displayed to remain on the target area. Taking snowflake elements as an example, snow particles can be displayed to remain on the target area and be deposited on the target area. It can also display deformed images of weather elements moving in the interface. For example, it can display deformed weather elements bouncing on the target area. Taking rain line elements as an example, it can display raindrops bouncing on the target area.”), determining the main element on a display interface ([0023] “After obtaining the current display position of the moving weather element in the interface, the method further includes: [0024] When the display position is the same as the edge position, it is determined that the distance between the display position and the target area meets the preset condition. [0025] Based on this implementation, the beneficial effects include at least the following: by obtaining the edge position of the target area, and depending on whether the current display position of the weather element is the same as the edge position, the deformed image of the weather element is displayed. When the weather element collides with the edge of the target area, the deformed image of the weather element will be displayed around the target area, making the interface display more vivid”), determining a line collision body corresponding to the main element and determining collision body information corresponding to the line collision body ( see at least [0219] of Simon “ (1.3) In the first frame of the first animation, the deformed weather element is located at the collision position. [0220] The collision location refers to the point where a moving weather element in the interface collides with the target area. It can be seen as the intersection between the movement trajectory of the weather element and the edge of the target area. By having the characteristic (1.3), in the subsequent display of the first animation, since the display position of the deformed weather element in the first frame of the first image can be regarded as the starting point of the bouncing motion, the display effect of the deformed weather element starting to bounce with the collision position as the starting point can be achieved.”; [0055] of LIN “In a standard face image, the coordinates of the points A, B, C and D, and values of a and b are all known. Therefore, relative positions of the center point C relative to the reference points A and B and relative lengths of the long and short axes of the ellipse relative to the reference points A and B may be recorded by using the above four linear difference coefficients. Where λ.sub.1 and λ.sub.2 are respectively used to record the relative positions of the center point C relative to the reference points A and B, λ.sub.3 is used to record the relative length of the long axis of the ellipse, and λ.sub.4 is used to record the relative length of the short axis of the ellipse, and the generated parameter of the special effect includes at least the four linear difference coefficients. In the above embodiment, since the special effect is elliptical, two difference coefficients are required to respectively record relative lengths of the long and short axes. However, in practice, the special effect may have an irregular shape, then the size of the special effect may be represented by an outer frame of the special effect. The outer frame of the special effect may be a minimum rectangle containing the special effect. Specifically, four straight lines may be formed by successively connecting four points located at an outermost periphery of the special effect, and a right-angled rectangle surrounded by the four straight lines is referred to as a minimum rectangle, and a center of the minimum rectangle is taken as a center point of the special effect. In this way, the relative position and relative size of the special effect may be indicated by using the above four linear difference coefficients regardless of the shape of the special effect. The parameter of the special effect may further include a rotation center, which may be directly represented by a feature point, and in this case, it is only required to record the number of the feature point. For a rotation center other than the feature point, the rotation center may be recorded in the above manner for recording a center point. Specifically, the rotation center and the center point may be superposed.”) and determining relative present information between each first present element and the main element based on the collision body information and current display information of each first present element (see at least [0160] of Simon “The algorithm can obtain the vertex coordinates of the edge image, take the coordinates of the points adjacent to the vertex coordinates in the edge image, obtain the slope between the vertex coordinates and the point coordinates, and determine whether the slope belongs to a preset slope range. If the slope belongs to the preset slope range, the point coordinates are taken as the point coordinates at the top of the slope. Similarly, the algorithm continues to take the next point coordinates adjacent to the vertex coordinates in the edge image, obtain the slope between the next point coordinates and the next point coordinates, and determine whether the slope belongs to a preset slope range. If the slope belongs to the preset slope range, the next point coordinates are taken as the point coordinates at the top of the slope. This process is repeated until the slope between the current point coordinates and the previous point coordinates does not belong to the preset slope range. The preset slope range indicates the range of the slope of the line, which can be greater than 0 and less than infinity. [0161] For example, please refer to Figure 9. Suppose the ordinate of the edge position corresponding to "1" is y3. We can take the coordinates of any point A(x1,y1) on the edge position, and take the coordinates of the next adjacent point B(x2,y2). Then the slope between the two points is K = (y2–y1)/(x2–x1). If k is greater than 0 and not infinite, then record the coordinates of point B. Then continue to take the adjacent point C of point B. By repeating the above process, each point can be used as the coordinates of the top of the slope of "1".;[0062] of LIN “There is a mapping relationship between the special effect on a standard face image and the special effect on a first face image collected by an image sensor, and depending on the mapping relationship, the special effect may include a fixed special effect and a tracking special effect. In an embodiment, a fixed special effect is generated, which is formed simply, where it is only required to set an absolute position of an entire range of the special effect relative to the image sensor. In an implementation, a display device is arranged in a one-to-one correspondence in pixels relative to an image acquisition window of the image sensor, and a position of the special effect in the display device is determined, then the special effect is generated at a corresponding position of the image acquired by the image acquisition window of the image sensor. The special effect is processed simply and the operation is easy, and the parameters used in the implementation are all corresponding to the position of the image acquisition window. In another embodiment, in a case of generating a special effect image, a feature point of the standard face image in step S1 is firstly acquired, and a position of the special effect on the standard face image is determined based on the feature point. A first face image corresponding to the standard face image is recognized from an image acquired by an image sensor. The position determined on the standard face image is mapped to a position in the first face image. The special effect is generated on the first face image to generate a special effect image. In this implementation, a relative position of the special effect in the first face image is determined, and the special effect is always located at the relative position no matter how the first face image moves and changes, such that a tracking special effect is realized. In a typical application, the standard face image subjecting to a triangulation process has 106 feature points, and relative positions of the special effect relative to the feature points are used to determine a relative position of a range of the special effect in the face image. The face image acquired by a camera also subjects to the triangulation process, and if the face in the camera moves or rotates, the special effect can be fixed at the relative position of the face, realizing an effect of a tracking special effect”) In addition, the same motivation is used as the rejection for claim 1. Both Simon and LIN are understood to be silent on the remaining limitations of claim 4.
In the same field of endeavor, Fujioka teaches determining the main element on a display interface, determining a line collision body corresponding to the main element and determining collision body information corresponding to the line collision body (see at least [0029] A collision check is done whenever the calculations that control the action of the bubbles are recalculated. In an embodiment of the present invention, the calculations are recalculated each time the GUI is refreshed, which may be optimally set at 42 frames per second. Of course, the refresh rate may be reduced or increased depending on the capabilities of the system that is running the GUI and the resulting look at utility of the GUI. The collisions between bubbles and boundaries follows the nature of elastic collision; the directions of velocity after collision are tangent lines based off the line of collision and the line of intersection of the two objects. This calculation may be done in the following manner, without limitation:”) and determining relative present information between each first present element and the main element based on the collision body information and current display information of each first present element ( see at least[0028] In an exemplary embodiment of the present invention, the bubbles may be in motion, or in a constant state of motion, and may move in a random direction until they come into contact with another bubble or edge of the screen, or another pre-set boundary. The reaction of the bubbles from a collision with another bubble or edge may be similar to that of billiards balls. Although the mathematics behind the relative action of objects in a "billiards" format is well known to those skilled in the art, a sample of various particulars associated with the present invention is included in the code herein, and as will be indicated therefrom to those skilled in the art.: [0030] In an embodiment of the present invention, the friction constant for the bubbles may be held at zero with an assigned mass of 1, regardless of the size of the object. Using these variables, all objects, regardless of size, may act uniformly under the same conditions. Similarly, when a collision occurs, elasticity may be what affects the resulting "bouncing" velocity on both of the colliding objects. In an embodiment of the present invention, the border or edge elasticity may be approximately 0.5 with the bubble elasticity at approximately 0.3, by way of non-limiting example. The elasticity may be combined during a collision and used to calculate the coefficient of restitution for the resulting velocity.”; )
Therefore, it would have been obvious to one of ordinary skill in the art before effective filling date of the claimed invention to modify the method of displaying moving weather elements on the interface of Simon and LIN with line of collision as seen in Fujioka because this modification would check collision between bubbles and boundaries ([0029] of Fujioka)
Thus, the combination of Simon, LIN, Fujioka teaches further comprising: before displaying, in response to determining that the relative present information between the at least one first present element and the main element is the persistence presentation, the at least one first present element on the main element, determining the main element on a display interface, determining a line collision body corresponding to the main element and determining collision body information corresponding to the line collision body; and determining relative present information between each first present element and the main element based on the collision body information and current display information of each first present element.
Regarding claim 5, Simon, LIN, Fujioka teach the method according to claim 4, wherein determining the line collision body corresponding to the main element and determining the collision body information corresponding to the line collision body comprises:
determining at least two key points corresponding to the main element (see at least [0055] of LIN “In a standard face image, the coordinates of the points A, B, C and D, and values of a and b are all known. Therefore, relative positions of the center point C relative to the reference points A and B and relative lengths of the long and short axes of the ellipse relative to the reference points A and B may be recorded by using the above four linear difference coefficients. Where λ.sub.1 and λ.sub.2 are respectively used to record the relative positions of the center point C relative to the reference points A and B, λ.sub.3 is used to record the relative length of the long axis of the ellipse, and λ.sub.4 is used to record the relative length of the short axis of the ellipse, and the generated parameter of the special effect includes at least the four linear difference coefficients. In the above embodiment, since the special effect is elliptical, two difference coefficients are required to respectively record relative lengths of the long and short axes. However, in practice, the special effect may have an irregular shape, then the size of the special effect may be represented by an outer frame of the special effect. The outer frame of the special effect may be a minimum rectangle containing the special effect. Specifically, four straight lines may be formed by successively connecting four points located at an outermost periphery of the special effect, and a right-angled rectangle surrounded by the four straight lines is referred to as a minimum rectangle, and a center of the minimum rectangle is taken as a center point of the special effect. In this way, the relative position and relative size of the special effect may be indicated by using the above four linear difference coefficients regardless of the shape of the special effect. The parameter of the special effect may further include a rotation center, which may be directly represented by a feature point, and in this case, it is only required to record the number of the feature point. For a rotation center other than the feature point, the rotation center may be recorded in the above manner for recording a center point. Specifically, the rotation center and the center point may be superposed.”; [0065] In an embodiment, the standard face image is divided into multiple regions, such as an eye region, a nose region, a mouth region, a cheek region, an eyebrow region, and a forehead region. Each of the regions includes an optimized feature point, which is a more representative feature point selected by data analysis. The feature point represents the region in which the feature point is located. For example, if a feature point in the eye region is selected as a reference point, this indicates that the eye region is determined as a target region for forming a special effect. Multiple sub special effects may be formed for each region, and each sub special effect separately tracks the region in which it is located. The sub special effects are combined together to form a special effect. The advantage of the above operation is that the number of feature points is reduced, and it is unnecessary to select one feature point from multiple feature points as a reference point. In addition, all the feature points displayed to the user are optimized, and once the user selects a region, a feature point in the region is also selected. A large special effect may be split into multiple sub special effects, to reduce the difficulty in forming the special effect”).; and
determining the line collision body according to the at least two key points, recording point coordinate information of the corresponding key points on the line collision body, and determining the point coordinate information as the collision body information, wherein the line collision body comprises at least one collision line segment, and two endpoints of the collision line segment correspond to the key points (see at least Simon [0160] The algorithm can obtain the vertex coordinates of the edge image, take the coordinates of the points adjacent to the vertex coordinates in the edge image, obtain the slope between the vertex coordinates and the point coordinates, and determine whether the slope belongs to a preset slope range. If the slope belongs to the preset slope range, the point coordinates are taken as the point coordinates at the top of the slope. Similarly, the algorithm continues to take the next point coordinates adjacent to the vertex coordinates in the edge image, obtain the slope between the next point coordinates and the next point coordinates, and determine whether the slope belongs to a preset slope range. If the slope belongs to the preset slope range, the next point coordinates are taken as the point coordinates at the top of the slope. This process is repeated until the slope between the current point coordinates and the previous point coordinates does not belong to the preset slope range. The preset slope range indicates the range of the slope of the line, which can be greater than 0 and less than infinity. [0161] For example, please refer to Figure 9. Suppose the ordinate of the edge position corresponding to "1" is y3. We can take the coordinates of any point A(x1,y1) on the edge position, and take the coordinates of the next adjacent point B(x2,y2). Then the slope between the two points is K = (y2–y1)/(x2–x1). If k is greater than 0 and not infinite, then record the coordinates of point B. Then continue to take the adjacent point C of point B. By repeating the above process, each point can be used as the coordinates of the top of the slope of "1".[0220] The collision location refers to the point where a moving weather element in the interface collides with the target area. It can be seen as the intersection between the movement trajectory of the weather element and the edge of the target area. By having the characteristic (1.3), in the subsequent display of the first animation, since the display position of the deformed weather element in the first frame of the first image can be regarded as the starting point of the bouncing motion, the display effect of the deformed weather element starting to bounce with the collision position as the starting point can be achieved. [0221]In one possible implementation, the first animation with the above characteristics can be displayed by performing the following steps one through three: [0222] Step 1: Based on the bouncing motion parameters, obtain the position of the weather elements in the first image of each frame. [0223] Bouncing motion parameters can be preset to indicate the bouncing motion trajectory. These parameters may include one or more of the following: velocity, acceleration, starting direction, and decay factor. For any two adjacent first images in the first animation, the displacement of the weather element during the display of the two first images can be obtained based on the bouncing motion parameters and the time interval between displaying these two first images. Based on the display position of the weather element in the previous first image and the displacement, the display position of the weather element in the next first image can be obtained. And so on, the display position of the weather element in each first image can be obtained based on the bouncing motion parameters and the display position of the weather element in the previous first image. By obtaining the display position of the weather element in this way, and drawing the weather element at the display position obtained in the first image of each frame, it can be ensured that the first animation has the above-mentioned characteristics (2), that is, the position change of the weather element in any two adjacent first images conforms to the bouncing motion trajectory.”;[0065] of LIN “ In an embodiment, the standard face image is divided into multiple regions, such as an eye region, a nose region, a mouth region, a cheek region, an eyebrow region, and a forehead region. Each of the regions includes an optimized feature point, which is a more representative feature point selected by data analysis. The feature point represents the region in which the feature point is located. For example, if a feature point in the eye region is selected as a reference point, this indicates that the eye region is determined as a target region for forming a special effect. Multiple sub special effects may be formed for each region, and each sub special effect separately tracks the region in which it is located. The sub special effects are combined together to form a special effect. The advantage of the above operation is that the number of feature points is reduced, and it is unnecessary to select one feature point from multiple feature points as a reference point. In addition, all the feature points displayed to the user are optimized, and once the user selects a region, a feature point in the region is also selected. A large special effect may be split into multiple sub special effects, to reduce the difficulty in forming the special effect.”; Fujioka: [0028] In an exemplary embodiment of the present invention, the bubbles may be in motion, or in a constant state of motion, and may move in a random direction until they come into contact with another bubble or edge of the screen, or another pre-set boundary. The reaction of the bubbles from a collision with another bubble or edge may be similar to that of billiards balls. Although the mathematics behind the relative action of objects in a "billiards" format is well known to those skilled in the art, a sample of various particulars associated with the present invention is included in the code herein, and as will be indicated therefrom to those skilled in the art. [0029] A collision check is done whenever the calculations that control the action of the bubbles are recalculated. In an embodiment of the present invention, the calculations are recalculated each time the GUI is refreshed, which may be optimally set at 42 frames per second. Of course, the refresh rate may be reduced or increased depending on the capabilities of the system that is running the GUI and the resulting look at utility of the GUI. The collisions between bubbles and boundaries follows the nature of elastic collision; the directions of velocity after collision are tangent lines based off the line of collision and the line of intersection of the two objects. This calculation may be done in the following manner, without limitation ..[0030] In an embodiment of the present invention, the friction constant for the bubbles may be held at zero with an assigned mass of 1, regardless of the size of the object. Using these variables, all objects, regardless of size, may act uniformly under the same conditions. Similarly, when a collision occurs, elasticity may be what affects the resulting "bouncing" velocity on both of the colliding objects. In an embodiment of the present invention, the border or edge elasticity may be approximately 0.5 with the bubble elasticity at approximately 0.3, by way of non-limiting example. The elasticity may be combined during a collision and used to calculate the coefficient of restitution for the resulting velocity.”) In addition, the same motivation is used as the rejection for claim 4.
Regarding claim 18, Simon and LIN teach the method according to claim 13, further comprising: determining display information of the corresponding first present element to be processed in the next video frame according to the target velocity information of each first present element to be processed and normal vector gravity information of the belonging line collision body; determining, based on the preset present mode, display information of each first present element in the non-stack display; and determining, based on the display information, relative present information between each main element and each first present element in the next video frame (see at least Simon [0212] Design 1: Display the first animation, which is used to show the deformed weather element bouncing on the target area. [0213] The first animation can be called a bouncing animation or a bouncing feedback effect. The first animation's display effects include: after a weather element collides with a target area, it deforms and bounces on the target area. For example, if the weather element is a rain line element, the first animation will display the following effects: after the rain line element collides with the target area, it will transform into raindrops and bounce on the target area. If the weather element is a hail element, the first animation will display the following effects: after the hail element collides with the target area, it will transform into hail particles and bounce on the target area.[0214] The first animation may include multiple frames of images. For the sake of distinction, any frame of the first animation will be referred to as the first image. Each first image includes deformed weather elements. The characteristics of the first image in the first animation may include the following (1.1) to (1.3): [0215] (1.1) The deformed weather elements in the first image of each frame are on the target area. [0216] The deformed weather element can be located on the upper surface of the target area, or it can be located above the target area but at a certain distance from the upper surface of the target area. With the feature (1.1), during the subsequent continuous display of multiple frames of the first image, it is possible to present the effect of deformed weather elements always bouncing above the target area, such as bouncing from the upper surface of the target area to the top of the target area, and then bouncing from the top of the target area to the upper surface of the target area, thereby simulating the phenomenon in the real world that when an object bounces, it will generally bounce above the plane due to the influence of gravity. [0217] (1.2) The positional changes of weather elements in any two adjacent first images conform to the trajectory of bouncing motion. [0218] The trajectory of the bouncing motion can include: take-off - rising position - reaching the highest point - falling position - reaching the lowest point. Correspondingly, the position change of the weather element can include: the weather element starts on the surface of the target area, then the display position of the weather element gradually rises, and when the position of the weather element reaches the highest point, the position of the weather element gradually falls until the weather element returns to the surface of the target area.”). Simon and LIN are understood be silent on the remaining limitations of claim 18.
In the same field of endeavor, Fujioka teaches normal vector gravity information of the belonging line collision body (see at least [0029] A collision check is done whenever the calculations that control the action of the bubbles are recalculated. In an embodiment of the present invention, the calculations are recalculated each time the GUI is refreshed, which may be optimally set at 42 frames per second. Of course, the refresh rate may be reduced or increased depending on the capabilities of the system that is running the GUI and the resulting look at utility of the GUI. The collisions between bubbles and boundaries follows the nature of elastic collision; the directions of velocity after collision are tangent lines based off the line of collision and the line of intersection of the two objects. This calculation may be done in the following manner, without limitation:… [0030] In an embodiment of the present invention, the friction constant for the bubbles may be held at zero with an assigned mass of 1, regardless of the size of the object. Using these variables, all objects, regardless of size, may act uniformly under the same conditions. Similarly, when a collision occurs, elasticity may be what affects the resulting "bouncing" velocity on both of the colliding objects. In an embodiment of the present invention, the border or edge elasticity may be approximately 0.5 with the bubble elasticity at approximately 0.3, by way of non-limiting example. The elasticity may be combined during a collision and used to calculate the coefficient of restitution for the resulting velocity. The calculation may be done as illustrated below, without limitation:
TABLE-US-00002 //ca.vn = vector normal of collision for object A //cb.vn = vector normal of collision for object B //pa = object A //pb = object B //te = combined elasticity // calculate the coefficient of restitution based on the mass, as the normal component var vnA:Vector = (cb.vn.mult((te + 1)).plus(ca.vn.mult(1- te))).divEquals(1); var vnB:Vector = (ca.vn.mult((te + 1)).plus(cb.vn.mult(1 - te))).divEquals(1)”) In addition, the same motivation is used as the rejection for claim 4.
Thus, the combination of Simon, LIN, Fujioka teaches further comprising: determining display information of the corresponding first present element to be processed in the next video frame according to the target velocity information of each first present element to be processed and normal vector gravity information of the belonging line collision body; determining, based on the preset present mode, display information of each first present element in the non-stack display; and determining, based on the display information, relative present information between each main element and each first present element in the next video frame.
3. Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Simon et al, IDS, CN109241465 (English translated) (“Simon”) in view of LIN et al, U.S Patent Application Publication No.2021/0366163 (“LIN”) further in view of Fujioka et al., U.S Patent Application Publication No. 2010/0281408 (“Fujioka”) further in view of WU et al, U.S Patent Application Publication No.20210158021 (“WU”)
Regarding claim 6, Simon, LIN, Fujioka teach the method according to claim 5, wherein determining the at least two key points corresponding to the main element comprises: in response to the main element corresponding to at least one part of a target user on the display interface, determining at least three key points of each part based on a key point recognition algorithm, so as to construct the line collision body based on the at least three key points (see at least Simon [0143] A screen coordinate system can be established to obtain the coordinates of each pixel in the edge image in the screen coordinate system, resulting in multiple point coordinates. One or more point coordinates can be selected from these multiple point coordinates and used as the edge position of the target area. The screen coordinate system can be as shown in Figure 4. The origin of the screen coordinate system is located at the upper left corner of the screen. The positive direction of the x-axis of the screen coordinate system is to the right along the screen, and the positive direction of the y-axis is to the down along the screen. [0144] For example, assuming the temperature text is "28°", the edge image of the temperature text is shown in Figure 5. The coordinates of each white pixel in Figure 5 in the screen coordinate system can be obtained to get a continuous segment of point coordinates. One or more point coordinates can be selected from this continuous segment of point coordinates as the edge position of the temperature text. [0145] Regarding the specific process of selecting point coordinates, in one possible implementation, the point coordinates that match the shape of the top of the target region can be obtained from the edge image and used as the edge position of the target region. [0158] For example, please refer to Figure 8. For the second type of characters -, 4, 5, 7, assume that the vertical coordinate of the edge position of the second type of character is represented as y2. The value range of y2 can be ymin, where ymin is the point with the smallest vertical coordinate in the edge image. [0159] (3) When the target area has a sloping top, obtain the coordinates of at least one point of the sloping top of the target area in the edge image as the edge location. [0160] The algorithm can obtain the vertex coordinates of the edge image, take the coordinates of the points adjacent to the vertex coordinates in the edge image, obtain the slope between the vertex coordinates and the point coordinates, and determine whether the slope belongs to a preset slope range. If the slope belongs to the preset slope range, the point coordinates are taken as the point coordinates at the top of the slope. Similarly, the algorithm continues to take the next point coordinates adjacent to the vertex coordinates in the edge image, obtain the slope between the next point coordinates and the next point coordinates, and determine whether the slope belongs to a preset slope range. If the slope belongs to the preset slope range, the next point coordinates are taken as the point coordinates at the top of the slope. This process is repeated until the slope between the current point coordinates and the previous point coordinates does not belong to the preset slope range. The preset slope range indicates the range of the slope of the line, which can be greater than 0 and less than infinity. [0161] For example, please refer to Figure 9. Suppose the ordinate of the edge position corresponding to "1" is y3. We can take the coordinates of any point A(x1,y1) on the edge position, and take the coordinates of the next adjacent point B(x2,y2). Then the slope between the two points is K = (y2–y1)/(x2–x1). If k is greater than 0 and not infinite, then record the coordinates of point B. Then continue to take the adjacent point C of point B. By repeating the above process, each point can be used as the coordinates of the top of the slope of "1".; [0055] of LIN “ In a standard face image, the coordinates of the points A, B, C and D, and values of a and b are all known. Therefore, relative positions of the center point C relative to the reference points A and B and relative lengths of the long and short axes of the ellipse relative to the reference points A and B may be recorded by using the above four linear difference coefficients. Where λ.sub.1 and λ.sub.2 are respectively used to record the relative positions of the center point C relative to the reference points A and B, λ.sub.3 is used to record the relative length of the long axis of the ellipse, and λ.sub.4 is used to record the relative length of the short axis of the ellipse, and the generated parameter of the special effect includes at least the four linear difference coefficients. In the above embodiment, since the special effect is elliptical, two difference coefficients are required to respectively record relative lengths of the long and short axes. However, in practice, the special effect may have an irregular shape, then the size of the special effect may be represented by an outer frame of the special effect. The outer frame of the special effect may be a minimum rectangle containing the special effect. Specifically, four straight lines may be formed by successively connecting four points located at an outermost periphery of the special effect, and a right-angled rectangle surrounded by the four straight lines is referred to as a minimum rectangle, and a center of the minimum rectangle is taken as a center point of the special effect. In this way, the relative position and relative size of the special effect may be indicated by using the above four linear difference coefficients regardless of the shape of the special effect. The parameter of the special effect may further include a rotation center, which may be directly represented by a feature point, and in this case, it is only required to record the number of the feature point. For a rotation center other than the feature point, the rotation center may be recorded in the above manner for recording a center point. Specifically, the rotation center and the center point may be superposed. [0029] of Fujioka “A collision check is done whenever the calculations that control the action of the bubbles are recalculated. In an embodiment of the present invention, the calculations are recalculated each time the GUI is refreshed, which may be optimally set at 42 frames per second. Of course, the refresh rate may be reduced or increased depending on the capabilities of the system that is running the GUI and the resulting look at utility of the GUI. The collisions between bubbles and boundaries follows the nature of elastic collision; the directions of velocity after collision are tangent lines based off the line of collision and the line of intersection of the two objects. This calculation may be done in the following manner, without limitation”) In addition, the same motivation is used as the rejection for claim 4. Simon, LIN, Fujioka are understood to be silent on the remaining limitations of claim 6.
In the same field endeavor, WU teaches wherein determining the at least two key points corresponding to the main element comprises: in response to the main element corresponding to at least one part of a target user on the display interface (see at least [0047] Taking that the virtual special effect is added merely based on the face key point detection technology as an example, after the face key point detection is performed on a face image, a lip area may be fitted based on a face key point detection result as shown in FIG. 2. Since the user face in the face image is not occluded by the occluder, the virtual lipstick special effect can be completely fitted to the lip area. However, with respect to FIG. 3, after performing the face key point detection and fitting a lip area based on the face key point detection result, the virtual lipstick special effect is finally rendered on the user's finger since the user's face in the face image is occluded by the finger.”) determining at least three key points of each part based on a key point recognition algorithm, so as to construct the line collision body based on the at least three key points (see at least [0038] Face key point detection is also known as face key localization or face alignment, and it refers to locating key areas of the face, including eyebrows, eyes, nose, mouth, facial contours, and the like, in a given face image. [0039] The set of key points is usually called shape. The shape includes location information of the key points, and the location information may generally be expressed in two manners. The first manner is locations of the key points relative to the entire image, and the second manner is locations of the key points relative to the rectangular frame of the face, which is not specifically limited in the embodiment of the present disclosure. [0078] As shown in FIG. 7, this step mainly aims to fit a corresponding location area requiring the virtual special effect processing based on the face key point detection result and the face part (such as, eyes, mouth and the like) the user wants to beautify, such that the location area may be processed by the virtual special effect to complete beautification of the face image in the first step. [0079] Generally, each virtual special effect matches a face part, such that when the user selects a corresponding virtual special effect, the face part that matches the virtual special effect is determined accordingly. For example, the eyebrow special effect matches the eyebrows, which indicates that the eyebrow special effect shall be added to the location of the eyebrows of the face; and the lipstick special effect matches the lips, which indicates that the lipstick special effect shall be added to the location of the lips of the face.[0080] In some embodiments, acquiring the first fusion image by fusing the selected virtual special effect and the face part matched in the target face image based on the face key point detection result includes, but is not limited to: determining the face part matching the selected virtual special effect; fitting a target location area in the target face image based on the face key point detection result and the determined face part; and acquiring the first fusion image by fusing the selected virtual special effect and the target location area of the target face image.”)
Therefore, it would have been obvious to one of ordinary skill in the art before effective filling date of the claimed invention to modify the method of displaying moving weather elements on the interface of Simon, LIN and Fujioka with face key point detection as seen in WU because this modification would fuse a virtual special effect and a face part matched in the target face image based on a face key point detection result ([0006] of WU)
Thus, the combination of Simon, LIN, Fujioka and WU teaches wherein determining the at least two key points corresponding to the main element comprises: in response to the main element corresponding to at least one part of a target user on the display interface, determining at least three key points of each part based on a key point recognition algorithm, so as to construct the line collision body based on the at least three key points.
4. Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Simon et al, IDS, CN109241465(English translated) (“Simon”) in view of LIN et al, U.S Patent Application Publication No.2021/0366163 (“LIN”) further in view of Fujioka et al., U.S Patent Application Publication No. 2010/0281408 (“Fujioka”) further in view of WU et al, U.S Patent Application Publication No.20210158021 (“WU”) further in view of Xu et al., U.S Patent Application Publication No.20200334875 (“Xu”)
Regarding claim 7, Simon, LIN, Fujioka and WU teach the method according to claim 6, wherein the at least one part comprises at least one of a plurality of parts of a face (see at least [0065] of LIN “In an embodiment, the standard face image is divided into multiple regions, such as an eye region, a nose region, a mouth region, a cheek region, an eyebrow region, and a forehead region. Each of the regions includes an optimized feature point, which is a more representative feature point selected by data analysis. The feature point represents the region in which the feature point is located. For example, if a feature point in the eye region is selected as a reference point, this indicates that the eye region is determined as a target region for forming a special effect. Multiple sub special effects may be formed for each region, and each sub special effect separately tracks the region in which it is located. The sub special effects are combined together to form a special effect. The advantage of the above operation is that the number of feature points is reduced, and it is unnecessary to select one feature point from multiple feature points as a reference point. In addition, all the feature points displayed to the user are optimized, and once the user selects a region, a feature point in the region is also selected. A large special effect may be split into multiple sub special effects, to reduce the difficulty in forming the special effect. [0075] of WU “For the face key point detection, the face key points include, but are not limited to, eyebrows, eyes, nose, mouth, face contour, or the like. In some embodiments, the model training may be performed based on a plurality of face images and face key point coordinates detected in the plurality of face images to obtain a network model having face key point detection capabilities. In the subsequent process, while detecting a face key point in a certain image, the image may be input into the network model to perform the face key point detection on the image based on the network model, thereby determining the face key point coordinates in the image.”) In addition, the same motivation is used as the rejection for claim 1. Simon, LIN, Fujioka and WU are understood to be silent on the remaining limitations of claim 7.
In the same field of endeavor, Xu teaches wherein the at least one part comprises at least one of a plurality of parts of a face and a plurality of parts of limbs and torso (see at least [0062] Exemplarily, the to-be-stroked area of the target object, for example, includes, but is not limited to: at least part of an object segmentation boundary area of an image where the target object is located, or a fitted line of at least two key points of the target object. [0063] The key points of the target object are pre-defined. In one implementation mode of the embodiments of the present disclosure, the key point includes at least one of the following: a head key point, a face key point, a shoulder key point, an arm key point, a gesture key point, a waist key point, a leg key point, a foot key point, a human skeleton key point, or the like. 0064] In one optional example, the head key point, for example, includes, but is not limited to, at least one of the following: a head top key point, a nose tip key point, a chin key point, or the like. [0065] In one optional example, the face key point, for example, includes, but is not limited to, at least one of the following: a face contour key point, an eye key point, an eyebrow key point, a nose key point, a mouth key point, or the like.)
Therefore, it would have been obvious to one of ordinary skill in the art before effective filling date of the claimed invention to modify the method of displaying moving weather elements on the interface of Simon, LIN, Fujioka and WU with detecting key point plurality of parts of a face and a plurality of parts of limbs and torso as seen in Xu because this modification would achieve the expected benefits of providing the spatial data needed to model human pose, expressions, and gestures.
Thus, the combination of Simon, LIN, Fujioka, WU and Xu teaches wherein the at least one part comprises at least one of a plurality of parts of a face and a plurality of parts of limbs and torso.
5. Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Simon et al, IDS, CN109241465 (English translated) (“Simon”) in view of LIN et al, U.S Patent Application Publication No.2021/0366163 (“LIN”) further in view of HE, U.S Patent Application Publication No.20230334801 (“HE”)
Regarding claim 8, Simon and LIN teach the method according to claim 1, wherein the main element is an eye part in a face (see at least [0065] of LIN “In an embodiment, the standard face image is divided into multiple regions, such as an eye region, a nose region, a mouth region, a cheek region, an eyebrow region, and a forehead region. Each of the regions includes an optimized feature point, which is a more representative feature point selected by data analysis. The feature point represents the region in which the feature point is located. For example, if a feature point in the eye region is selected as a reference point, this indicates that the eye region is determined as a target region for forming a special effect. Multiple sub special effects may be formed for each region, and each sub special effect separately tracks the region in which it is located. The sub special effects are combined together to form a special effect. The advantage of the above operation is that the number of feature points is reduced, and it is unnecessary to select one feature point from multiple feature points as a reference point. In addition, all the feature points displayed to the user are optimized, and once the user selects a region, a feature point in the region is also selected. A large special effect may be split into multiple sub special effects, to reduce the difficulty in forming the special effect.”); and
wherein the method further comprises: before stacking and displaying, in response to determining that the relative present information between the at least one first present element and the main element is the persistence presentation (see at least [0053] of Simon “Based on this implementation, the beneficial effects include at least the following: each frame of the second image will contain new weather elements in addition to the weather elements in the previous second image, and the new weather elements are located at the previous deposition height. As the rendering process is executed, the height of the weather elements drawn in each frame of the second image becomes higher and higher. Thus, when multiple frames of the second image are played continuously, the display effect of the weather elements on the target area being stacked higher and higher is presented, which realistically simulates the deposition phenomenon.”), the at least one first present element on the main element, determining an absence of a first present element being stacked on the eye part in response to determining that the eye part according to the key points of the eye part (see at least [0080] of Simon “Based on this implementation, the beneficial effects include at least the following: by attenuating the transparency of the weather element, the transparency of the weather element in multiple interfaces will gradually decrease. Therefore, when multiple interfaces are displayed in succession, the weather element can be presented as gradually disappearing from the interface from solid to blurred. By attenuating the length and width of the weather element, the length and width of the weather element in multiple interfaces will gradually decrease. Therefore, when multiple interfaces are displayed in succession, the weather element can be presented as decreasing in size from large to small.”; [0187] Based on this, the display position of the weather element in each interface can be obtained according to the initial position, initial velocity, acceleration and initial angle of the weather element. For each interface, the weather element will be drawn at the display position of the weather element in the interface. Since the display position of the weather element is different in multiple interfaces, the effect of the weather element moving in the interface can be displayed by continuously displaying multiple interfaces. [0188] Optionally, the transparency, length, and width of the weather element can be attenuated according to the attenuation degree of the weather element to obtain the transparency, length, and width of the weather element in each interface. In each interface, a weather element with the corresponding transparency, length, and width is drawn. Since the transparency of the weather element gradually decreases in multiple interfaces, by continuously displaying multiple interfaces, the display effect of the weather element gradually disappearing from the interface can be presented. Since the length and width of the weather element gradually decrease in multiple interfaces, by continuously displaying multiple interfaces, the display effect of the weather element gradually shrinking in the interface can be presented.”; [0065] of LIN “In an embodiment, the standard face image is divided into multiple regions, such as an eye region, a nose region, a mouth region, a cheek region, an eyebrow region, and a forehead region. Each of the regions includes an optimized feature point, which is a more representative feature point selected by data analysis. The feature point represents the region in which the feature point is located. For example, if a feature point in the eye region is selected as a reference point, this indicates that the eye region is determined as a target region for forming a special effect. Multiple sub special effects may be formed for each region, and each sub special effect separately tracks the region in which it is located. The sub special effects are combined together to form a special effect. The advantage of the above operation is that the number of feature points is reduced, and it is unnecessary to select one feature point from multiple feature points as a reference point. In addition, all the feature points displayed to the user are optimized, and once the user selects a region, a feature point in the region is also selected. A large special effect may be split into multiple sub special effects, to reduce the difficulty in forming the special effect.”) In addition, the same motivation is used as the rejection for claim 8. Both Simon and LIN are understood to be silent on the remaining limitations of claim 8.
In the same field of endeavor, HE teaches wherein the main element is an eye part in a face ([0034] With the above technical solution, the anchor point of the 3D special effect element to be attached to the face is determined, and the attachment point on the 3D facial model to which the 3D special effect element is to be attached is determined; the anchor point of the 3D special effect element is then attached on the attachment point in a way that the anchor point overlaps the attachment point; the anchor point is moved to the center of the world coordinate system of the 3D facial model, and the 3D effect element is bound with the 3D facial model. Thereby, the 3D special effect element is joined to the surface of the 3D facial model, and the 3D special effect element can universally fit any user face, and can move with and follow a face action in real time when the face moves. Hence, an offset of the 3D special effect element during a head rotation is avoided, and a real-time interaction between the 3D special effect element and the face action is improved. For example, when opening and closing eyes, the 3D special effected false eyelashes may move with the eyes synchronously and rotate with an orientation of eyelids. In addition, in this technical solution, feature points namely attachment points of the 3D facial model are obtained, and an interaction between the 3D special effect element and the face can be expanded. For example, an effect that a spider climbs out of the mouth and climbs slowly on the face to the back of the head can be realized, and other procedural animations such as a face growth animation, a fluid animation, and the like can be realized.”); and wherein the method further comprises: before stacking and displaying, in response to determining that the relative present information between the at least one first present element and the main element is the persistence presentation (see ate least [0036] In addition, the 3D special effect element to be attached may be configured with a random dimension and a random direction. Hence, in an example where the 3D special effect element is snowflakes falling over the hair, such configuration can realize that many snowflakes with random dimensions fall over the hair in random directions. [0037] In addition, the method for reconstructing a facial model according to the embodiments of the present disclosure may be used in combination with a particle system, so as to realize more three-dimensional special effects. The particle system is commonly used in games for instantiating a large number of model scenes such as snowflakes and flames, so as to reduce the number of renderings. Taking snow on the face as an example, through the combined use of the method according to the embodiments of the present disclosure and the particle system, it is possible to realize a 3D special effect of an accumulation of snow over time, or a special effect that a decoration, such as an icing, is attached to the face”), the at least one first present element on the main element, determining a first present element being stacked on the eye part, in response to determining that the eye part is in a closed state according to the key points of the eye part (see at least [0038] FIG. 3 is a schematic block diagram of a device for reconstructing a facial model according to an embodiment of the present disclosure. Reference is made to FIG. 3, the device for reconstructing a facial model includes: an anchor point determination module 31, configured to determine an anchor point of a 3D special effect element to be attached on a face; an attachment point determination module 32 configured to determine an attachment point on a 3D facial model to which the 3D special effect element is to be attached, where the 3D facial model corresponds to the face; a superposition module 33, configured to attach the anchor point to the attachment point in a way that the anchor point overlaps the attachment point; and a binding module 34, configured to move the anchor point to a center of a world coordinate system of the 3D facial model, and bind the 3D special effect element with the 3D facial model. [0039] With the above technical solution, the anchor point of the 3D special effect element to be attached to the face is determined, and the attachment point on the 3D facial model to which the 3D special effect element is to be attached is determined; the anchor point of the 3D special effect element is then attached on the attachment point in a way that the anchor point overlaps the attachment point; the anchor point is moved to the center of the world coordinate system of the 3D facial model, and the 3D effect element is bound with the 3D facial model. In this way, the 3D special effect element is attached to the surface of the 3D facial model, so that the 3D special effect element can universally fit any user face, and can move with and follow a face action in real time when the face moves. Hence, an offset of the 3D special effect element during a head rotation is avoided, and a real-time interaction between the 3D special effect element and the face action is improved. For example, when opening and closing eyes, the 3D special effected false eyelashes may move with the eyes synchronously, and rotate with an orientation of eyelids. In addition, in this technical solution, feature points namely attachment points of the 3D facial model are obtained, and an interaction between the 3D special effect element and the face can be expanded. For example, an effect that a spider climbs out of the mouth and climbs slowly on the face to the back of the head can be realized, and other procedural animations such as a face growth animation, a fluid animation, and the like can be realized.”)
Therefore, it would have been obvious to one of ordinary skill in the art before effective filling date of the claimed invention to modify the method of displaying moving weather elements on the interface of Simon and LIN with detecting closing and opening eyes of HE because this modification would move special effected with the eyes synchronously ([0034] of HE)
Thus, the combination of Simon, LIN and HE teaches wherein the main element is an eye part in a face; and wherein the method further comprises: before stacking and displaying, in response to determining that the relative present information between the at least one first present element and the main element is the persistence presentation, the at least one first present element on the main element, determining an absence of a first present element being stacked on the eye part, in response to determining that the eye part is in a closed state according to the key points of the eye part.
Allowable Subject Matter
Claims 9-11, 14-17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Regarding claim 9, the prior art of record taken alone or in combination, fails to disclose or render obvious: “wherein determining the relative present information between each first present element and the main element based on the collision body information and the current display information of each first present element comprises: determining, for each first present element and according to position information of the current first present element, a distance value between the current first present element and the corresponding line segment in a normal vector direction of the corresponding collision line segment on the line collision body, and determining whether there is an intersection point between a projection point of the current first present element in the normal vector direction and the collision line segment, wherein the normal vector direction is determined according to key point information corresponding to the two endpoints of the collision line segment; and in response to determining that the distance value is less than a preset collision distance threshold and there is the intersection point, determining that the relative present information between the current first present element and the corresponding line collision body is the persistence presentation.”
Claims 10-11 are objected based on objection of claim 9.
Regarding claim 14, the prior art of record taken alone or in combination, fails to disclose or render obvious: “wherein determining the target velocity information of each first present element to be processed after colliding with the main element comprises: for each first present element to be processed, determining relative velocity information to be updated based on a movement velocity of a current first present element to be processed and an average movement velocity of a corresponding collision line segment in a line collision body, wherein the average movement velocity is determined according to position information of two endpoints of the collision line segment in a previous video frame and position information of the two endpoints of the collision line segment in a current video frame; determining, based on the relative velocity information to be updated, a first projection component of the relative velocity information to be updated in a normal vector direction of the line collision body; updating the relative velocity information to be updated based on the first projection component to obtain velocity information to be used of the current first present element to be processed; and determining target velocity information of the current first present element to be processed according to the velocity information to be used and the average movement velocity of the line collision body.”
Claim 15- 17 are objected based on objection of claim 14.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SARAH LE whose telephone number is (571)270-7842. The examiner can normally be reached Monday: 8AM-4:30PM EST, Tuesday: 8 AM-3:30PM EST, Wednesday: 8AM-2:30PM EST, Thursday and Friday off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at (571) 272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SARAH LE/Primary Examiner, Art Unit 2614
Prosecution Insights